Do not merge that agent change yet: the six-layer regression verification baseline for vibe projects

An agent just changed your code. What is your first instinct? If the answer is "merge it and see how it runs," you have already outsourced verification to your users. The faster a vibe project moves, the more that habit costs: one change touches three places, the login page gets fixed, the checkout page quietly breaks, and a mobile layout collapses at one particular width — and you only find out when a user sends a screenshot. The fix is not "manually click through everything more carefully." It is a regression verification baseline: a fixed set of checks that must pass before any agent change merges. Not finished, no merge. Not passing, no excuses — fix it.

A baseline is not a test suite: keep three things apart

A regression baseline, a unit test suite, and manual acceptance are three different things, and mixing them up lets each one dodge responsibility. Unit tests answer "does this function compute correctly?" Manual acceptance answers "does this feature feel right to use?" The regression baseline answers a third question: "did this change break something that used to work?" Its subject is not new features but old ones: login, signup, checkout, search, export, notifications — the parts users rely on every day and you long assumed were fine.

That distinction sets the baseline's size: it does not need to cover every code path, only the paths whose breakage users would notice immediately. A small project with five core flows might have a baseline of twenty checks; a product with ten modules might have sixty. Size is not the goal. Stability is: a set of checks that produces the same verdict every time beats a hundred checks that flicker.

The six layers: from fast to slow, from cheap to expensive

Layer one, static gates: type checking, lint, and formatting must all pass before anything else runs. This layer is the fastest and least controversial, and the agent can run it locally. Wire it into the pull request's automated checks; a request that fails static gates does not even deserve human review yet.

Layer two, unit and contract tests: unit tests for core functions, plus contract tests for the interfaces between modules. Contract tests exist for a classic vibe-project failure: an agent changes the data shape module A returns, module B still parses the old shape, each unit suite passes, and the combination breaks. One contract test per key interface is enough.

Layer three, golden-path end-to-end: automated scripts (or a fixed manual step list) walking every core business flow — signup, login, creation, payment (in sandbox), export, notification delivery. The number of golden paths equals the number of core flows: no more, no less. Each path needs an explicit "done" signal — for example, the order status becomes paid, or the exported file opens cleanly — not a vague "the page loads."

Layer four, data and migration verification: when a change touches the database, the baseline must include a migration rehearsal — upgrade a test database from the old version to the new one, roll back, upgrade again, with no data loss and old data still readable. It must also include a "dirty data" check: run core flows against realistic messy data (overlong titles, empty fields, special characters, timezone edges), because agent-written code most often passes on clean test data and fails on real data.

Layer five, visual and layout spot checks: screenshot comparisons for key pages, at one desktop width and one mobile width. This layer does not need whole-site screenshots — only the screens users see first: the homepage, the core action page, the checkout page. Set the visual threshold loosely; it exists to catch collapsed layouts, overlapping elements, and vanished buttons, not pixel-level jitter.

Layer six, performance smoke: caps on core API response time, homepage load time, and the number of database queries a single action triggers. Agents easily introduce N+1 queries while "fixing the feature"; performance smoke exists to catch that regression before merge, not after users complain about slowness.

Four iron rules for running the baseline

Rule one: the baseline is a merge gate, not a suggestion list. A request that has not finished the baseline does not merge. This applies equally to humans and agents; the moment you allow one "merge now, backfill later" exception, the baseline is dead.

Rule two: when the baseline fails, distinguish fixing the baseline from fixing the code. If the baseline itself is wrong (for example, an expectation went stale), fix the baseline and note why. If the code broke an old feature, fix the code. The worst move is quietly loosening the baseline to turn it green — loosening a baseline requires a separate review explaining why the old expectation no longer holds.

Rule three: flaky failures get a name and a record. Any check that fails intermittently must be logged the same day: what failed, what the rerun showed, the suspected cause, and who owns it. An unlogged flaky failure will be waved away next time as "that old problem again" — and that time it may be a real problem.

Rule four: the baseline grows with the product, one item at a time. When a new core flow ships, add one golden path with it; when an old flow retires, remove one. The baseline file itself is versioned, and baseline changes go through the same review process as code changes.

The verification brief for an agent: not just "run the tests"

When you hand an agent the baseline, the brief should read like a checklist, not a slogan. A good brief includes: which commands to run (verbatim), the pass criteria for each command, what to do first on failure (paste the failure log verbatim, pinpoint the file and line, judge whether it is a code problem or a baseline problem), and which layers to rerun after a fix (at minimum the failed layer and the directly related layer above it). It must also state one prohibition: the agent may not change baseline expectations to get a pass; changing a baseline needs separate human approval.

One practical device is a "baseline report template": every change ships with a table, one row per layer, marked pass, fail, or not applicable, with a log summary on any failing row. Reviewers read that table before the code diff. Without it, reviewers can only guess how big a change's impact is — and gut feeling is the least reliable review tool at vibe-development speed.

Boundaries with existing guides

This piece has a clear division of labor with four nearby guides. The agent-damage rescue guide (cycle102) covers what to do after a change breaks things — freeze, triage, and recover piece by piece. That is after the fact; this piece is about stopping breakage before merge. They form a before-and-after pair, not a repetition. The visual regression guide covers the tooling of screenshot comparison; this piece treats visual spot checks as just one of six layers and does not expand on tool selection. The AI evals guide measures the quality of model output itself; this piece measures a change's damage to existing features — a different object. The error-monitoring guide covers finding and locating errors after release; this piece covers not shipping them in the first place. Merging all four into one "quality assurance mega-guide" would blur every checklist; kept separate, the pre-merge baseline can be executed as written.

Start this week: begin with twenty checks

• Step one (1 hour): list your core flows (usually three to six) and write one golden-path check per flow; run them first as a manual step list, then automate gradually.
• Step two (at the next merge): wire static gates and unit tests into the pull request's automated checks — no pass, no review.
• Step three (within the month): add the remaining four layers — contract tests, migration rehearsal, visual spot checks, and performance smoke — one item each to start, no greed.
• Step four (ongoing): log flaky failures, route baseline loosening through separate review, and attach the baseline report template to every merge request.

Bottom line: the speed advantage of vibe development should not rest on users discovering regressions for you. Set a baseline that must run before merge, and let every agent change first prove "the old features are still alive" before arguing how good the new feature is.