Skip the Demo, Read the Repo: Judge an AI-Generated Project in 10 Minutes

New AI-generated projects surface every day: GitHub trending, Vibe Jam entries, the "built this app in 3 hours" link in your feed. The demos are all silky, the READMEs all breathless. The question: which ones deserve your time to use, your fork to build on, your recommendation to others?

My take: skip the demo, read the repo. An AI-generated project's trustworthiness can be checked in 10 minutes — and you're not looking for pretty code, you're looking for traces of humans. This checklist comes from reviewing hundreds of AI-generated projects. Every item covers what to look at, how, and what good looks like.

Why first: "it runs" and "it's trustworthy" are far apart

The ZCode incident of September 18, 2026 is the perfect footnote: Z.ai's AI coding tool was caught quietly packaging entire workspaces for upload — .git history included. The tool ran fine, ran well even — but nobody could call it trustworthy. Days later Z.ai was forced to open-source ZCode under Apache 2.0 to stop the bleeding. The lesson for everyone: if even a big vendor's AI tool does things in the dark, imagine what a random vibe project off the street might do.

Another dimension is the Moltbook incident (covered on this site): the AI-only social network acquired by Meta three months after launch — cool story, but early versions had a data leak. The silkier the demo, the harder you should ask: what did the silk cost? Did anyone review it?

So the checklist's underlying logic: AI is responsible for "making the thing," humans for "proving it's trustworthy." No human traces — no trust.

There's a deeper reason: 2026's AI is too good at looking trustworthy. It writes gorgeous READMEs, adds CI badges (even when CI is red), drops phrases like "enterprise-grade architecture." Demo-driven evaluation is broken — you have to go down into the repo and inspect like an engineer. This checklist is the inspection manual: ignore the ads, check the goods; ignore the demo, read the repo.

Minutes 1–3: look for "human-ness" — git history and README

1. Is git history one giant dump or rhythmic iteration? Glance at commits. If the whole project is 1–2 massive commits ("initial commit" with 500 files), the AI barfed it out in one go and the author never reviewed it — because no human can review changes they never broke down. Good signals: dozens of small commits with human rhythm ("fix: …", "add: …", "refactor: …"), however sloppy the messages. Human-ness isn't in pretty messages; it's in thinking in segments. Also check the time distribution: 40 commits at 3 AM one night versus 40 commits across two weeks tell completely different stories.

2. Does the README have a "Known Issues" section? My most valued item. AI-generated READMEs are always "feature list + install steps + roadmap," brimming with confidence. But real projects have sharp edges: a browser incompatibility, an API quota, an unfinished feature. An author who dares write "Known Issues / Limitations" has actually run it and bled on it. Conversely, a flawless README paired with one-shot git history basically reads "AI generated, shipped unreviewed" — it runs, but nobody owns it.

Minutes 4–7: look for "engineering" — tests, dependencies, error handling

3. Any tests and CI? Doesn't take many — even a few covering core flows. An AI's willingness to write tests approximates the author's quality bar. Check for .github/workflows and whether CI is green. A project without CI is one whose author can't promise "it'll still run after the next change" — would you bet on it? Note: test files can be AI placebos too. Skim for real assertions versus assert True comfort blankets.

4. Are dependencies sane? Open package.json / requirements.txt / go.mod and count. A whole UI library for one button? Three HTTP clients for one request? AI has a bad habit: it doesn't know "good enough," only "this library solves it." Dependency bloat means bigger attack surface, slower builds, nastier upgrades later. Good signal: lean dependencies, each heavy one justified (README or comment says why this one). Bad signal: 800 packages in node_modules for a to-do list.

5. Is error handling real or try/except pass? Search for catch / except / .catch(. Classic AI smell: the happy path flows like poetry, error branches are all empty — catch (e) {}, or a lonely console.log(e). Real-world code spends 20–30% of its lines on errors. If you can barely find error handling, the author (and the AI) only verified the sunny-day case — and sunny days are the rarest days in production.

6. Any over-engineering smell? The other AI extreme: microservices for a blog, Redis plus a message queue for three config values. AI doesn't understand YAGNI — it stacks every "best practice" it knows. Rule of thumb: architecture complexity ÷ business complexity > 3 warrants suspicion. Good projects are "just enough"; AI projects are often "over-armed" or "naked," rarely "just right."

Minutes 8–10: look for "red lines" — security, license, reproducibility

7. Scan for secrets and hardcoding. Search api_key, secret, password, sk-. AI routinely writes sample keys straight into code; worse, real keys get committed. .env.example present is a good signal; .env committed is instant disqualification. Also check .gitignore exists and is sane — a project without one was authored by someone who never thought about "what mustn't enter the repo."

8. Is there a LICENSE? "Open source" without a license legally means "all rights reserved" — your fork or commercial use could infringe. AI never adds licenses unprompted; an author who didn't bother signals casualness about "shipping." MIT/Apache-2.0 good; missing or "TBD" bad.

9. Does the demo open, and does it match the repo? Click the demo link if there is one, and compare against the latest commit time: a three-month-old demo with yesterday's commits means demo and code have likely forked. More direct: pick a prominent demo feature, search the repo for its code — can't find it, and the demo is a "special build." Happens in jam entries: a demo branch polished for competition while main stays half-baked.

10. Issue tracker and commit pulse. Final glance: do issues get replies? When was the last commit? A "live" project, however small, shows patching activity; a "dead" one is a tombstone no matter how pretty the README. Distinguish "stable" from "dead": a tool untouched for months may be stable (no bugs is the best news), but an app silent for half a year is probably abandoned.

Walkthrough: running one project through it (simulated)

Items aren't enough; here's a full simulated 10 minutes. You find on GitHub: "AI podcast clipper — upload audio, auto-extract highlights," 800 stars, breathless README. Clock starts:

Minute 1: commits — 2 total, "initial commit" and "update readme." Red flag: one-shot dump, no segmented thinking by the author.

Minute 2: README — full feature list, clear install steps, but no Known Issues, no architecture notes. Yellow: heavy AI smell, though the install steps look genuine (verifiable later).

Minutes 3–5: package.json — 87 dependencies, three audio libraries doing the same job. Red: bloat. Tests — no test dir, no CI. Red.

Minutes 6–7: search catch — 12 hits, 9 empty or bare console.log. Red: sunny-day coder. Search sk- — clean, no hardcoded keys, .env.example present. Green.

Minute 8: LICENSE — missing. Red. Demo link — opens, matches README screenshots. Green, with the fork caveat noted.

Minutes 9–10: issues — 17 open, 0 replies; last commit 4 months ago. Red: abandoned.

Verdict: 6 of 10 red. Play with the demo, but don't fork it, don't recommend it, don't build on it — nobody owns it. Ten minutes just saved you ten future hours. That's the checklist's value: it doesn't buy correctness, it buys time.

What good looks like

Hold Arkai against the list: MIT-licensed on GitHub (item 8 ✓), About page honestly disclosing AI operation (item 2's human-ness ✓), daily updates showing live commits (item 10 ✓). The Great Taxi Assignment's credits disclose the toolchain plainly (item 2 ✓). These aren't necessarily the prettiest codebases, but the human traces are thick — that's where trust comes from.

Intellectual honesty: the checklist falsifies, it doesn't verify

Honesty requires stating the boundaries. First, it's good at finding untrustworthy, bad at proving trustworthy — a project passing all 10 can still hide a deep logic bug; one failing 5 of 10 is almost certainly unowned. Second, demos and early prototypes shouldn't face production standards — many jam entries are one-shot commits with no tests, and their goal is "fun," not "maintainable"; applying this list to them misses the point. Third, the list expires: AI gets better at faking human-ness every year (it writes "Known Issues" sections now), so the checklist needs annual updates.

The right mindset: use it as a fast falsifier — 10 minutes, more than 3 strikes, set it aside; all clear, then go deep. Its value isn't a trustworthiness certificate; it's spending your limited time on projects worth the deep dive.

One-line summary

AI drove the cost of "making things" to zero, so "judging whether things are trustworthy" became the scarce skill. The whole list is one sentence: look for human traces — segmented commits, honest READMEs, real tests, living issues. AI can generate code; it can't generate the smell of "someone owns this." Ten minutes following the smell dodges 80% of pitfalls. The remaining 20% is your own taste — and taste comes from reading 100 projects, not 100 guides. Go find an AI-generated project and run the list on it right now; one practice run beats ten readings. After ten projects you'll notice yourself getting faster — number eleven might take three minutes.