The Model Is Only a Tenth: Stack Overflow on "Agent = Model + Harness"

First, a clarification: this isn't news — it's an opinion essay

What we're covering today comes from the Stack Overflow official blog, published October 7 — the sixth installment of their "operating system for coding agents" series. It's not a product launch, not a funding round, not a new benchmark record. It's a method-driven opinion piece with a clear engineering stance. But it deserves to be read as seriously as news, because it carries both a provocative number and an observation that anyone who has used a coding agent heavily will immediately recognize.

The provocative number: on the public Terminal-Bench 2.0 evaluation, a LangChain coding agent climbed from outside the top 30 all the way into the top 5 — from 52.8% to 66.5%, a full 13.7 percentage points. The method: the model didn't change one bit; everything changed was the "harness" around the model — system prompt, tool selection, middleware like self-verification and loop detection. The recognizable observation: the first session with a coding agent feels crisp — it reads your code, makes a clean change, writes a test. By the thirtieth session, something has rotted: it re-discovers files it has already seen, contradicts a decision it made three weeks ago, guesses a test command off the top of its head, and quietly stops writing tests — because nobody forced it to.

The thesis compresses into one line: "the agent is the model plus its harness" — and "the model is maybe a tenth." The other nine tenths — prompts, tools, context policies, hooks, sandboxes, sub-agents, observability — decide whether you get working software at the end of a quarter. Examined honestly, most agent failures aren't model failures; they're configuration failures: a missing tool, a vague rule, an absent guardrail, a context window stuffed with noise. So when quality decays across a long project, the lever is almost never "wait for a smarter model." It's the harness — and unlike the model, the harness is yours to build.

For our readers, the value of this piece is that it distills the daily pain of vibe coders into an engineering methodology you can copy. It also happens to complete a trilogy with two pieces we've already published: our MCP client integration guide covered connecting capabilities to agents ("plugging in"), our Evals piece covered verifying quality ("testing") — this one covers keeping an agent trustworthy over time ("raising"). Together: plug in, test, raise. Also worth noting alongside this: our cycle 40 story on Agnes AI's open-source Agnes Harness talked about execution-layer infrastructure — a different front of the same war as this essay's "harness engineering."

13.7 points: same model, only the harness changed

Let's dwell on this number, because it's easy to misread. Terminal-Bench 2.0 is a public benchmark of terminal tasks, testing what agents can actually accomplish in a real command-line environment. LangChain's agent went from outside the top 30 to the top 5 on this board, gaining 13.7 points. The qualifier the original post keeps hammering: the model didn't change at all. The change list had exactly three items: system prompt, tool selection, and middleware like self-verification and loop detection.

Why does this number matter? Because in the 2026 AI discourse, the performance narrative is almost monopolized by model vendors: every benchmark refresh means "a new generation of model." This case proves the counter-narrative: the same model, a different harness, and the performance ceiling moves up by double digits. For indie developers, the implication is concrete — you don't have a model-training budget, but you have full freedom to rewrite prompts, tune tools, and add checkpoints. Those 13.7 points are, in other words, a "poor man's model upgrade," available to everyone.

The deeper meaning is a shift in how we read benchmarks. The subtext of the essay: if a benchmark's result depends heavily on harness configuration, then "how strong is this model" and "how strong is this agent" are two entirely different questions. An agent that looks brilliant on one task may simply have a harness tuned for that task type; if it falls over in a different setting, that doesn't mean the model is bad. Next time you see a "so-and-so agent tops the leaderboard" story, ask first: is the model strong, or is the harness well-tuned? That distinction is the first practical judgment this essay hands its readers.

The four rots: why the thirtieth session goes stupid

The essay breaks an agent's long-term decay into four concrete pathologies, each paired with a prescription. This "symptom → prescription" structure is the most imitable part of the whole piece, because it turns the vague "agents are unreliable" into fixable configuration problems.

The first pathology is discipline: the agent skips tests and quietly expands scope. Prescription: a constitution file, read every session. The second is context: the agent keeps re-learning the repo. Prescription: tiered, hook-maintained CONTEXT.md files. The third is memory: corrected mistakes get repeated. Prescription: durable per-fact memory files. The fourth is review: one blind spot applied uniformly everywhere. Prescription: a multi-lens agent review panel.

The ordering matters: it follows the lifecycle of working with an agent — first discipline (know how to behave), then context (know what this project is), then memory (remember past lessons), finally review (someone to poke holes). For solo developers, the second and third hurt the most: you've definitely watched an agent ask for the third time "how do I run the tests in this repo," and you've definitely watched it step into a hole you corrected last week. The prescriptions for those two — tiered CONTEXT.md and durable memory — are exactly what we'll unpack below, and the parts of this essay you can most directly copy into your own projects.

Artifact 1: the constitution — a few non-negotiables, read every session

The "constitution" is a lean, always-loaded file stating the non-negotiables: the workflow loop (Plan → Document → Implement → Test → Review → Commit); quality bars (tests in the same diff, no dead code, conventional commits); adapter discipline (no vendor SDK in domain code).

The hard constraint is "keep it short." This file gets read on every turn, so every line costs context budget. A constitution only states non-negotiables — not an operations manual. That's a genuinely useful design principle: plenty of people write AGENTS.md files that run to thousands of lines, cramming in guides, conventions, and history, and the agent ends up less focused in long sessions, not more. The essay's stance is blunt — a constitution is a legal code, not an encyclopedia. If it doesn't fit on one page, it doesn't deserve the name.

The direct takeaway for vibe coders: if you already keep an AGENTS.md in your project, give it a "constitutionalization" pass — keep only the clauses that would make you swear if violated, and move the tutorial content elsewhere. Less is more, here literally: every line saves context.

Artifact 2: the autonomy boundary — free rein on the reversible, gates on the irreversible

This is, in my view, the single highest-leverage decision in the essay: let the agent run free on everything reversible, and pause only on the two things that aren't — pushing to a remote, and deploying. Expressed as a permissions config: allowed are reads, edits, writes, searches, git add/commit, running tests and builds; require confirmation are git push, npm publish, aws, terraform apply, kubectl apply; outright denied are git push --force and anything with --no-verify.

The elegance is that it redefines the boundary of "autonomy." Most people pick one of two extremes: make the agent ask permission at every step and exhaust themselves, or give it full rein and one night discover it force-pushed main. The essay's split is by reversibility. Local code changes, test runs, commits to local branches — all reversible, let it rip; push and deploy — irreversible, that's where the human checkpoint goes. The agent can then work for an hour and commit a dozen times while you only review at the push.

There's an engineering footnote that's easy to miss: the essay warns you to verify your runner's match semantics — prefix vs. substring matching differ — or your --no-verify/--force denials and npm publish interception may never actually trip; broad globs like npm:* quietly allow npm publish. A classic "thought I had a guardrail but didn't" trap. If you copy this permissions config, test one denial rule in your own environment to make sure it fires; don't trust the config file's good looks.

Artifact 3: tiered CONTEXT.md — 200 lines at the root; stale context is a bug

One giant context file rots and blows the budget. The essay's answer is tiering: the root CONTEXT.md is the map — what this is, how to run and test it, a one-line module index, invariants, where state lives — capped around 200 lines, read at session start; per-module CONTEXT.md files hold responsibility, key files, dependencies, gotchas, and how to test just that module — read before editing it; DECISIONS.md is append-only — date, decision, why, alternatives rejected.

The sharpest rule in the whole essay: "stale context is a bug, fixed in the same change that invalidated it" — not "later." It turns documentation maintenance from a conscience-driven chore into a build red line. Think of the usual pattern: you change a module's responsibility, the CONTEXT.md still describes the old one; the agent reads the stale description next session and makes a wrong call, and the root cause was your un-updated change three months ago. Making "update the context" a mandatory part of the same diff as the code change is worth more than any tool.

Practical advice for solo projects: start with just the root CONTEXT.md, 200-line cap, covering three things — how to run, how to test, the module index. Split out per-module files only when the project outgrows one file. You can start appending to DECISIONS.md today — its full form (the decision log shared by multiple agents) gets its own treatment below, but in the single-agent phase it's simply a "why did we decide this" memo that future-you will thank present-you for.

Artifact 4: durable memory — answering "what breaks if I change this?"

Context describes the code; memory holds what the code can't tell you across sessions. The essay's structure: a memory/ directory where MEMORY.md is the index (one line per fact), each fact gets its own file (typed as user preference, feedback, project, or reference), plus a graph.jsonl representing modules and decisions as nodes in {from, rel, to} triples, with relations like depends-on, adapts, and supersedes.

The graph exists to answer the question agents are worst at: "what breaks if I change this?" That's my favorite detail in the essay. It names the capability gap of current coding agents precisely: agents are good at "understanding what this code does" and terrible at "inferring the blast radius of a change." Human engineers maintain a dependency map in their heads; agents wake up amnesiac every session. Writing that map down and persisting it is like grafting onto the agent an organ it can't grow on its own.

For vibe coders, this part's ROI may be underrated. You don't need the full graph on day one — start by recording "holes": every time the agent steps in one and you correct it, write down "don't do it this way + why" as one memory. Next time it reaches for the same mistake, the memory intercepts it first. Compounded over months, your agent becomes "the one that knows your project" — a project-specific rapport no model upgrade can buy.

Artifact 5: the multi-lens review panel — five lenses, one verdict

Instead of having the agent review its own work (which is the same blind spot checking itself), run focused sub-agents in parallel, each wearing one lens, with strictly uniform output: product (does it solve the stated problem? scope creep? — at plan stage), architect (correctness, data model, failure modes, coupling), infra (resource limits, deploy/rollback, resilience), security (authz, input validation, secrets, audit trail), test engineering (is the suite real, or happy-path theater? — on every change).

Each reviewer emits: [BLOCKER|MAJOR|MINOR] (confidence%) — file:line — issue — fix; then a VERDICT. Only high-confidence BLOCKERs and MAJORs actually gate; the confidence threshold keeps the panel from drowning you in nits.

Two judgments here are worth savoring. First, diversity of review matters more than reviewer "intelligence" — five mediocre reviewers with different lenses catch more than one brilliant reviewer with a single lens. That matches human code-review experience: the most valuable comment often comes from the person who doesn't know your domain but does know security or performance. Second, the confidence threshold is the anti-noise key. A multi-agent review without thresholds degenerates into competitive nitpicking, burying the real BLOCKERs under dozens of MINORs. For solo developers, start with two lenses — test engineering on every change, security when auth or external input is involved — and add more once the loop is smooth.

Artifact 6: the PostToolUse hook — automating "remember to update the docs"

This is the piece that glues the other five together: a hook that fires after every edit. Wire it in settings.json, pointed at a tiny script: after each Edit/Write/MultiEdit, the script reads the tool event on stdin and emits a checklist as context — "did this change alter module responsibilities/key files/gotchas? then update the nearest CONTEXT.md; did it touch dependencies or APIs? then add an edge to memory/the graph; did it set a precedent? then log a line in DECISIONS.md; did it produce a durable fact? then update memory. Make these edits in THIS change — stale context is a bug."

The key design choice: the hook only reminds, never silently rewrites — the agent makes those updates in the same diff, all reviewable and visible. That contrasts with "auto-tidying doc" tools that edit your docs while you aren't looking; here the reminder becomes context, the execution stays with the agent, and the diff stays yours to see.

This is also the most engineering-flavored move in the essay: it admits a reality — "remember to update the docs" doesn't work on willpower, and doesn't work on spot-checking in code review either. Only a mechanism that fires automatically after every edit works. For vibe coders, this is the cheapest, fastest-payback artifact of the six: a script of a few dozen lines, solving the ancient problem of "docs always three versions behind the code." If you only copy one of the six, copy this one.

The full loop for one agent: six artifacts and a cycle

The workflow tying the six together: session start reads the root CONTEXT.md and MEMORY.md; each task runs plan → panel reviews the plan → you approve → implement subtask (tests in the same diff) → panel reviews the diff → full test+lint → commit → hook refreshes context/memory/graph → next; the only stops are push and deploy.

The essay's summary: that's the whole harness for one agent — six artifacts and a loop. Each is a few lines of config or a markdown file on its own; together they're the difference between an agent that's brilliant for an afternoon and one that's still trustworthy on the thousandth edit.

That "thousandth edit" framing deserves a pause. We evaluate coding agents almost entirely on first-session performance — the dazzling demo, the high benchmark score. But a real project's costs land on the 30th and 300th sessions: by then, what matters isn't how clever the model is, but whether the harness can hold back entropy. The essay effectively proposes a new evaluation dimension: how long is an agent's "shelf life"? Next time you pick an agent tool, alongside "how impressive is the first run," ask: does it have any mechanism that keeps it from going stupid three months in?

Multi-agent coordination: don't let them "chat" — let them "read and write the same world"

One agent, properly harnessed, stays productive for months. But run several on the same repo at once and a new failure mode appears: they collide — editing the same files, undoing each other's work, re-litigating decisions another agent already made. The instinct is to have them "communicate." The essay argues against it: communication doesn't scale and isn't reliable. What works is the old trick human teams use to parallelize — coordinate through shared, written state, not conversation. Agents shouldn't message each other. They read and write a small set of shared files encoding the state of the work: who owns what, what's decided, what's claimed, what's done. Each agent acts on the world (the repo plus these files); the next one reads the world to decide what to do. Coordination becomes a property of the shared state, not of a conversation — which means it survives a restart and is auditable. The essay's metaphor: insects coordinate huge builds this way, by changing and reading a shared environment rather than holding meetings.

The shared state has four parts. An ownership map: who owns which paths, defaulting to directory ownership; editing someone else's turf or shared ground (like shared/contracts/**) requires a claim. Claim/release: cheap mutual exclusion with no lock server — write a CLAIM line before starting (timestamp, agent, task, paths, estimate) and a RELEASE line when done (timestamp, agent, task, commit). A progress log: append-only and timestamped; an agent starting up reads the last day to learn "what got done, who's working where." And a decision log: append-only, stopping agent-B from silently reversing agent-A's reasoned choice — load-bearing enough for its own section below.

The most honest passage is the essay's self-demolition of claim/release: it says outright that the mechanism is advisory/optimistic, not a lock — two agents can both scan, both see nothing, and both claim the same path (the classic TOCTOU race). If you need real mutual exclusion, use a primitive with atomic check-and-set: a uniquely-named git branch (the push fails if it already exists), or a conditional write to a coordination store. The file-append claim is good enough when collisions are rare and a human merges — "don't sell it as a lock." That habit of "propose a solution, then hand-label its limits" is a big part of why this essay is credible, and it's a part we shouldn't lose in retelling.

Three more anti-unraveling rules are equally practical: branch per task, humans merge (unreviewed changes never hit main; the merge is the human checkpoint); no agent-to-agent calls (kills hidden side-channels; all coordination stays in the auditable files); a stable public surface per module (stops parallel refactors from breaking each other). Plus two iron laws: decisions are append-only and binding; claim before touching shared or other-owned paths.

The decision log: append-only DEC-017, and "status is computed"

Every non-trivial system accumulates decisions: which datastore, agents only propose, all model traffic through one gateway, which component may loop and which may not. Six weeks later nobody remembers why, someone "fixes" one of them, and a load-bearing choice quietly reverses. The classic remedy is the architecture decision record. In an AI-assisted build — where agents make changes too — a decision log isn't just hygiene; it's the mechanism that keeps humans and agents from contradicting each other. A human team has hallway memory; agents don't. An agent picking up a task reads the log to learn the constraints it must honor — and won't reverse a choice whose reason it can't see.

The essay's example entry is DEC-017: "We route ALL model traffic through one gateway. Why: one place for rate-limits, cost tracking, the kill switch, and provider swaps. Alternatives rejected: per-service SDK calls (scatters cost + coupling everywhere)." Entries are short and structured, and the valuable parts are the "why" and the "rejected alternatives" — they're what stop the decision from being re-argued every quarter.

The subtler design is that "status is derived": to change your mind you never edit an old entry — you append a new one, e.g. DEC-031: "Supersedes DEC-017: the gateway also handles embeddings, not just chat." Each entry carries a supersedes pointer, and "superseded" isn't a stored field you flip — it's computed: an entry counts as superseded if and only if a later entry's supersedes points at it. The history — reversals included — is itself valuable: it tells the story of how the system's thinking evolved, and a half-remembered old entry is never silently wrong, only visibly superseded.

But a log nobody enforces rots. So wire it into review: a diff that violates an accepted decision gets rejected. The runnable handles are two fields: governs (which paths this decision constrains — without it the gate can't know which decisions apply to a diff) and forbids_imports (one cheap, fully machine-checkable violates() rule, like banning certain SDK imports; richer checks go to a human or an agent reviewer). Now the log isn't documentation that decays — it's an active constraint the build maintains. To change a decision, an agent writes a superseding entry and flags a human; it never silently codes around one.

The essay also gives a keep-in/keep-out list: log the non-obvious and hard-to-reverse — architecture, data stores, security postures, the boundaries of agent autonomy, anything you'd be annoyed to see quietly undone; keep out what the code already says plainly, transient implementation details, things that change freely without consequence, trivia (padding hides the important entries). The list itself is worth copying: most teams' ADRs die from logging everything until nobody reads them.

Our take: the model is becoming a commodity; differentiation is moving to the harness

Placed on the 2026 timeline, this essay reads as a correction to the first half of the year's narrative. H1 was about models: parameters, context lengths, benchmark scores; H2's weightiest moves are happening outside the model — execution layers, harnesses, observability, skill sedimentation. Read alongside our cycle 40 story on the Agnes Harness, the trend is clear: the model is commoditizing, and differentiation is migrating to the harness and the execution layer.

Three direct implications for vibe coding readers. First, don't judge a tool by its model: when evaluating a coding agent, ask three questions — does it have a durable memory mechanism? what's its context strategy? what do its review/guardrails look like? The answers predict your three-months-later experience better than "it's powered by model X." Second, start building your own harness today: a constitution file, tiered CONTEXT.md, a memory directory, a PostToolUse hook — none of them depend on any vendor and all of them can land in your project today. An agent that increasingly "knows your project" is a rapport no model swap can buy; it's the real moat of an indie developer. Third, evaluate everything by shelf life: a dazzling demo doesn't matter; being trustworthy on the thousandth edit does. For any agent tool, first check whether it has a mechanism against entropy.

A cooling note to close. This is an opinion essay, not a paper: the 13.7-point figure comes from LangChain's own Terminal-Bench 2.0 practice, and the original doesn't publish full experimental controls (no ablation of the harness components, for instance) — treat it as directional evidence, not a precise scientific conclusion. And the full six-artifact rollout isn't cheap: a multi-lens review panel multiplies token spend per change, so solo projects should scale to their means — start with the cheap, high-value pieces like the hook and the constitution. Opinions can be radical; rollout should be gradual.

But the direction, we think, holds: when everyone has access to comparably smart models, what separates outcomes is the nine tenths outside the model. And those nine tenths are something you can start building today.