Back to Explore
GuideVibeFix editorialUpdated Nov 11, 2026

Don't Discover the Breaking Change on Integration Day: Contract Testing for Vibe Projects

Unit tests pass on both sides and the integration still breaks — because each side is correct alone and wrong together. This guide explains consumer-driven contract testing: four elements of a contract, a 39-check file, a five-step Pact setup, and when not to use it.

Two developers sitting in front of screens reviewing code together, one pointing at the code on screen

First, let's get the name right, so you don't open the wrong article: contract testing here means the agreement between services about an API — the frontend says "I need this field," the backend says "I will always return this field," and both sides write the agreement into a file and check against it before changing the interface. It is not the same as the site's guide on freelance contracts, which is about the legal agreement you sign with a client; and it has a different job from the cycle170 end-to-end testing guide, which is about walking the whole user path. That one checks the journey; this one makes sure both sides agree on the deal before the journey starts. One judgment up front: AI is best at changing both sides at once, and best at producing changes where each side is correct on its own but wrong together. Contract testing is a cheap lock on that kind of change — and the price of not having the lock always shows up on integration day, or worse, on launch day.

Two developers sitting in front of screens reviewing code together, one pointing at the code on screen

Three ways to fail while all your unit tests are green

Here is a fictional example. The product "Pinecone List" and the developer "A-Yuan" below are fictional, and the numbers are made up to make the problem clear — they are not data from a real project. A-Yuan used AI to ship three features in one day: a to-do list, filtering by tag, and marking a to-do as done. The backend had 42 unit tests, all green. The frontend had 31 unit tests, all green. He clicked deploy. Then users started reporting errors. In the postmortem, each of the three failure modes had struck once:

Failure one: a renamed field nobody announced. The backend renamed dueDate to dueAt, because the AI thought the latter was more idiomatic. The frontend was still reading dueDate, which now came back as undefined, so the due-date column went blank everywhere. The backend tests stayed green — they only check that what the backend returns is correct, never what the frontend is reading.

Failure two: a silently drifting type. priority used to be the numbers 1, 2, 3. During a refactor the AI turned it into the strings "high", "medium", and "low". The frontend compared numbers with strings, and the sort order fell apart. This is the most infuriating kind of bug: the page does not crash, it just sorts wrong, and users take two days to notice — and assume they misremembered.

Failure three: optional turned required. The create-to-do endpoint used to accept an empty tag. The AI added a validation rule, and empty tags started returning 400. The old client, other people's shortcuts, and scheduled scripts all started failing on the same day. The API's "temperament" had changed, but the version number had not, the docs had not been updated, and the only signal was the error count.

All three share one root cause: the bug was not in either side's code; it was in the mismatch between what the two sides believed the interface to be. Unit tests check whether one side is internally correct. Integration tests check whether both sides work together. Contract testing is the cheapest kind of integration test — it does not boot the whole system, it just checks the agreement line by line.

What a contract actually is: four elements of consumer-driven testing

Contract testing has a qualifier that matters: "consumer-driven." It means the contract is written by the side that uses the API (the consumer, usually the frontend), not by the side that provides it (the provider, usually the backend). This is not academic fussiness; it settles who is responsible. The backend can refactor its internals however it likes, as long as it still satisfies the lines the frontend wrote into the contract. When the frontend needs a new field, it writes it into the contract first, and the backend implements it. In one sentence: whoever uses the API writes the contract.

A usable contract file has at least four elements, and missing any one of them will come back to hurt you some night:

  • Interaction: what one request looks like — method, path, headers, body. Write it precisely, not "roughly."
  • Expectation: what this request should return — status code, the fields that must be in the response body, and each field's type. Only write the fields the consumer actually uses. Every extra field you write is debt you owe yourself.
  • Provider state: what the data must look like before this contract runs — for example, "a to-do with id 7 exists." Without this, verification passes or fails at random.
  • Version: which version of the consumer this contract belongs to. Without versions, the contract files rot into a pile of paper nobody can tell apart.

There is a rough test for whether a contract is well written: print it out, hand it to someone who has never seen the code, and see if they can write a backend that passes verification from the contract alone. If they can, it is a good contract. If they cannot, the contract is full of "you know what I mean" — and "you know what I mean" is the first thing an AI refactor throws away.

One picture: what actually goes into a contract file

People writing their first contract usually make one of two mistakes: too thin — only checking that the status code is 200, which is the same as writing nothing — or too thick, copying the backend's whole database schema, so changing one field turns everything red. The contract file for our fictional example product "Pinecone List" has 39 checks in five categories, and you can copy this ratio directly:

39 checks in a contract file, by category

CategoryChecksWhat it verifies
Field presence12Not one field the consumer actually uses may go missing
Types9Number vs. string, timestamp as string or as epoch
Required vs. optional7Which fields may be empty, and whether empty means null or the field is omitted
Enum values6Fields like status and priority may only take the agreed values
Error codes5What comes back for not found, forbidden, and bad input

Notice the trade-off behind the ratio: field presence takes the biggest share (12 checks) because a missing field is the most common and most lethal integration failure; error codes get only 5, because error handling should stay coarse — write every exception into the contract and you will have to change the contract just to reword an error message. One more principle: a contract verifies "shape," not "business rules." That a to-do's title cannot be empty is a business rule — it belongs in a unit test. That the title field must be a string is shape — it belongs in the contract. When the shape is stable, the business logic can change freely; when the shape moves, both sides must know at the same time.

Where contract testing sits in the testing pyramid

People often ask how contract testing relates to other kinds of tests. One sentence settles it: unit tests keep one side correct, contract tests keep both sides in agreement, end-to-end tests keep the whole journey working. They do not replace each other; each covers its own stretch. This table is worth sharing with your team so you stop re-litigating "should we write this test" every time:

Test typeWhat it coversHow fast it runsWhat it cannot catch
Unit testWhether one function or module is internally correctMilliseconds; hundreds run in secondsTwo sides disagreeing about the interface
Contract testWhether the consumer and provider honor their agreementSeconds; no need to boot the whole systemWhether the whole business flow works
Integration testWhether several real services work connectedMinutes; needs databases and services upHow the system behaves when a third party is down
End-to-end testThe full journey from opening the page to finishing the taskSlowest; minutes per caseToo expensive to cover every interface combination

What the table shows is the trade-off: contract tests run almost as fast as unit tests while covering what only integration tests used to cover — that is why their return on investment is so high. They also have a ceiling: a contract can only guarantee that "the agreed fields are all correct," not that "the business flow is correct." If the login endpoint returns a token in the right format but the token expires after 5 minutes, a contract test will not catch it — you need the end-to-end case that logs in and acts again 10 minutes later. So the right mindset is: contract testing is not a replacement for the testing pyramid; it is the layer in the middle that most people skip. The typical vibe project has a pile of unit tests, two or three end-to-end tests, and a complete vacuum in between — and every interface change is discovered by manual integration testing and luck. Contract testing fills that vacuum.

One practical number helps you decide: if your project has one frontend and one backend that deploy together, contract testing adds little — shared TypeScript types are enough, because one type change breaks compilation on both sides at once. But the moment a second consumer appears — a second frontend, a mini-program, a caller of your public API, a scheduled script — contract testing goes from "nice to have" to "must have." Every additional consumer doubles the chance that two sides disagree about the interface, and the contract file is the only place where every consumer sees the same agreement.

Five steps to a working Pact setup in one afternoon

Pick Pact not because it is the trendiest tool, but because it is the de facto standard for consumer-driven contract testing, with complete docs and a large ecosystem — and the idea itself does not depend on it. You can get the same flow running with a handwritten JSON file and a small verification script. The five steps:

  1. The consumer writes the test first. The frontend (or caller) writes a test: I send this request, I expect that response. The test does not call the real backend; it calls a mock server that answers according to the contract.
  2. Generate the contract file. When the test passes, the mock server writes "what the request looks like, what the response should be" into a JSON file. That is the contract — one file, shared by both sides.
  3. The provider verifies against the contract. The backend starts up and runs each line of the contract against the real API. If it passes, the backend satisfies the frontend's expectations. If it fails, the error points at one exact item, one exact field.
  4. Put verification in CI. Every backend commit runs contract verification; every frontend contract change regenerates the file. If verification fails, the pull request goes red and cannot merge.
  5. Version the contract file. The contract file goes into the repository with tags, released alongside the code. When the frontend ships contract v2.3, the backend must pass v2.3 verification before v2.3 can go live.

Getting these five steps running in one afternoon is a modest bar: the first contract covers your single most important endpoint — usually the one the first page after login calls. Do not be greedy; get the pipeline moving first, then add endpoints one by one. My advice: cover money and login first, everything else second. When payment, subscription, or login breaks, it is an incident; when anything else breaks, it is a bug. The priorities differ by an order of magnitude.

What does a contract file look like? No need to imagine — here is the "get one to-do" endpoint from our fictional example "Pinecone List," in JSON, the core of it only a few dozen lines:

{
  "consumer": { "name": "songguo-web" },
  "provider": { "name": "songguo-api" },
  "interactions": [
    {
      "description": "Get one existing to-do",
      "providerState": "a to-do with id 7 exists",
      "request": {
        "method": "GET",
        "path": "/todos/7"
      },
      "response": {
        "status": 200,
        "body": {
          "id": 7,
          "title": "Buy milk",
          "done": false,
          "priority": 2,
          "dueAt": "2026-11-09T09:00:00Z"
        }
      }
    }
  ]
}

Three details matter when you read this JSON. First, providerState states the precondition — "a to-do with id 7 exists." Without that line, the backend's database is empty during verification, the test fails at random, and you lose an afternoon questioning your sanity. Second, the response lists only the 5 fields the frontend actually uses, while the backend really returns 14 — the other 9 are not in the contract, which means the backend may add or change them freely. Third, priority is the number 2, not a string, so the type is locked in here. If the AI later tries to turn it into a string, verification goes red and the error names priority exactly — expected a number, got a string — instead of vaguely saying "the API is broken."

There is one more use of the contract file that is specific to vibe projects: feed it to the AI. Before every interface change, paste the contract file into the conversation and tell the AI to "read the contract first, then change the code, then run verification." The AI reads a contract far more reliably than it reads your spoken description, because a contract is a machine-readable, precise description, while your spoken description is full of "you know what I mean."

When not to use contract testing: three cases where it hurts

Contract testing is not a silver bullet. In three situations it adds more trouble than it saves, and it is better to say so honestly than to push it anyway.

First, frontend and backend live in one repository and one AI session. If your project is a monolith — frontend and backend deploy together and roll back together — then an interface change hits both sides at once, in the same release, and contract testing buys you very little. Shared TypeScript types are enough here: one type change breaks compilation on both sides at the same time, faster than any contract file.

Second, the other side is a third-party API. When you call Stripe or OpenAI, the contract is written by you alone — the other side will not change to match it. The right approach there is snapshot testing against recorded real responses, or an anti-corruption layer that keeps the third party's uncertainty at your own front door, instead of pretending you can constrain them.

Third, the interface is still changing three times a day. In the first week of a product, when the API changes daily, the contract file changes more often than the code, and maintenance costs more than it saves. Wait until the interface has mostly settled — usually once the first real user starts using it — and then add contracts. Contract testing is a lock for "an interface someone already depends on," not for a sketch on scratch paper.

Versions and breaking changes: three tiers of handling

Contract testing earns its keep not on ordinary days, but on the day you need to change an interface. Every interface change falls into one of three tiers, each handled differently, and mixing them up is how incidents happen:

  • Adding a field (safest): the response gains one more field; old consumers ignore it. Just add it — no new version needed. Append one line to the contract file and verification still passes.
  • Changing what a field means (most dangerous): the field name stays the same but the meaning changes — for example, status stops meaning "to-do status" and starts meaning "sync status." The name did not change, so compilation does not fail and types do not fail; only a contract test catches it. The correct move: add the new field first, mark the old one deprecated, give consumers at least one version to migrate, and only then delete it.
  • Deleting a field, changing a type, making a field required (new version required): these three are breaking changes. Before shipping a new version, ask one question: is any consumer still on the old contract? If yes, run both versions in parallel. If no, write it in the changelog and delete it.

Here is a counterintuitive judgment: the biggest value of contract testing is not "catching errors" but "forcing the person who changes the interface to think clearly about whether this is a breaking change." When AI changes an interface, it never thinks about that — it just wants the code to compile. When contract verification goes red, a human is forced to stop and ask: am I adding a field, or changing a meaning? That pause prevents half of all incidents.

Three common pitfalls: how contract testing itself fails

Contract testing can fail too, and when it does it looks a lot like "the tests are being difficult." Knowing these three pitfalls in advance saves at least a week of friction:

Pitfall one: the contract is too thick. The classic beginner mistake is writing every field the backend returns into the contract, including ones the frontend never uses. Then the backend adds one internal field and the contract goes red; someone rewords an error message and the contract goes red again. After enough red, the team starts saying "let's turn off contract testing for now and ship" — and the lock is dead. The fix is simple: only write the fields the consumer actually uses. The test is easy — can you find this field in the frontend code? If not, do not write it.

Pitfall two: faked provider state. To save effort during verification, people return the same canned to-do no matter what id is asked for. Verification passes, but it is fake, and launch day still fails. Provider state must be real: before verification runs, write a row into the database that matches the state description, and clean it up after. Three extra lines of code, and the verification actually means something.

Pitfall three: nobody owns the contract file. The contract file gets generated, then lies in a corner of the repository. The frontend changes the interface without updating the contract, the backend stops running verification, and three months later the contract file and the real API are two different things. The fix is to treat the contract file as code: a commit that changes the interface must change the contract too, and code review looks at the contract diff before the code diff. A contract file is not documentation; it is code, and it deserves the same treatment — review, versions, CI, all of it.

12-item pre-launch checklist

Copy this, run through it, then ship:

  1. Your single most important endpoint (the one the first screen after login calls) is covered by a contract.
  2. The contract lists only the fields the consumer actually uses — not the backend's whole table.
  3. Every field's type is pinned down; time fields say whether they are strings or timestamps.
  4. Required vs. optional is written out, and both sides agree whether an empty value comes back as null or as a missing field.
  5. Enum values are fully listed, and adding a new enum value follows the "add first, deprecate later" flow.
  6. Error codes cover not found, forbidden, and bad input.
  7. Provider state (the precondition data) is written into the contract, so verification does not pass by luck.
  8. Contract verification is in CI — red means no merge and no deploy.
  9. The contract file is in the repository with tags, so it maps one-to-one onto code versions.
  10. Breaking changes went through the three-tier handling; nothing quietly changed meaning.
  11. Before the AI changes an interface it reads the contract, and after changing it runs verification — the flow is written into your development prompt.
  12. There is no "you know what I mean" in the contract file — a stranger could write a backend from it.

One last repeat of the core judgment: contract testing is not about writing more tests. It is about making "both sides agree" something you have verified before you press deploy, instead of discovering it after users report errors. Unit tests protect one side, end-to-end tests protect the whole journey, and contract tests protect the seam between the two sides — and in a vibe project, where the AI rewrites the interface eight times a day, the seam is exactly where things crack first.

Views 6Comments 0

Comments (0)

ME
0/1000
Loading comments...
Browse projectsPublish your project

Related articles

A developer types at a dual-monitor workstation; the left screen shows a dark code editor, the right a testing dashboard with green checkmarks and bar charts
News
Did Your Agent Get Better or Worse? AWS Open-Sources UAEF to Measure Agents from Every Framework with One Ruler

On 11 November 2026, AWS open-sourced UAEF: adapters turn traces from LangGraph, LangChain, Strands and Bedrock AgentCore into one format, then 36 metrics across 7 dimensions score them, flagging regressions past 5%. The judge model is hardened too — scores must cite verbatim evidence, and never decide safety.

Developer WorkflowProduct LaunchTesting & Quality
A modern data center corridor lined with black server racks, cable trays and ventilation ducts overhead, blue indicator lights blinking on the racks
News
Two Months Late, PostgreSQL 19 Is Finally Here: Repack Tables While They Run, and Make Standbys Wait for Your Writes

On 19 November 2026, PostgreSQL 19 shipped — two months late, after a flood of AI-reported vulnerabilities rewrote the schedule. It brings REPACK (CONCURRENTLY), pg_plan_advice, and WAIT FOR LSN: tables can be reorganized while they run, query plans can be pinned, and standbys wait for your writes.

Product LaunchData WorkflowsBackend Engineering