Claude Wrote Math Proofs for 123 Open-Source Libraries: RISC-V Assembly-Level Formal Verification, 180M Lines of Rocq, Near-Zero Human Review

The opening sentence of Anthropic's research blog post carries an unusual urgency: "Most software is riddled with bugs, which AI can now easily find and exploit."

That's not hype. In the same week, an OpenAI model helped surface the PHP2Shell vulnerabilities, and Anthropic itself had Claude "shrink-wrap" 25 open-source projects including OpenSSH — deleting unneeded code to reduce the attack surface. AI's ability to find bugs is compounding, while traditional defenses — testing, fuzzing, code audit — are all forms of sampling: you never know how many bugs are hiding in the corners you didn't check.

On October 22, Anthropic dropped something bigger. Working with Theorem, a formal-verification startup, they had Claude do what used to take elite expert teams years: write mathematical proofs for 123 real-world open-source libraries — at the RISC-V assembly instruction level — proving they can never crash. Anthropic's own characterization: "to our knowledge, this represents the first large-scale effort to use AI to apply formal verification to production software at the assembly code level."

Note the weight of that phrasing. Formal verification isn't "testing harder." It's a different game: not "we ran ten thousand inputs and nothing crashed, so it's probably fine," but "we have mathematically proven that no possible input can crash it." For the first time, AI security is graduating from "finding bugs" to "proving the absence of bugs."

How it works: translating fuzz tests into theorems

The pipeline design is elegant. Worth unpacking:

Step one: borrow an existing specification. The hardest part of formal verification is stating what "correct" means — writing a mathematical spec. Anthropic took a shortcut: Google's OSS-Fuzz corpus. Since 2016, OSS-Fuzz has continuously fuzz-tested open-source C/C++ projects; each test has an entry point with a simple rule: any input that crashes the program is a bug the authors care about. Anthropic translated that rule into a formal spec: "this test never crashes, for any input under 64 MB."

Step two: prove it at machine-code level. Each library and its fuzz test were compiled to a RISC-V executable (instrumented with AddressSanitizer, so improper memory access crashes). Theorem's formal semantics of the RISC-V instruction set and of Linux processes, plus the no-crash property, mechanically generated the proof obligation. Claude wrote the proofs — in Rocq (formerly Coq), a proof-assistant language.

Step three: machine-check everything. Rocq's proof checker (the kernel) verifies each proof line by line. Purely mechanical: it passes or it doesn't. No human needs to "believe" any argument.

Some numbers for scale: 239 entry points, 123 libraries, a few weeks, roughly 180 million lines of Rocq proof. A typical proof runs about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions — the proof is 30-plus times larger than the code. That ratio is exactly why formal verification has historically been ruinously expensive.

The crucial line, which Anthropic repeats: virtually none of those 180 million lines needs human review. Because checking is mechanical, there are only four things you must trust: the formal model of RISC-V, the formal model of Linux processes, the exact theorem statements proved, and the Rocq checker itself. That's it. The remaining 180 million lines stand or fall by machine judgment.

Why this was impossible before: seL4's twenty person-years

To feel the weight of this breakthrough, recall how expensive formal verification used to be. Anthropic cites the canonical example: the seL4 microkernel — two person-years to write the code, twenty person-years to formally verify it. Ten times the implementation effort, requiring globally scarce specialists.

That's why formal verification lived only where failure was unthinkable: Airbus avionics, military systems, crypto libraries. Ordinary open-source projects? Forget it — who's going to spend twenty person-years proving a JSON parser?

What LLMs change is the cost structure, not the principle. The 123 libraries Claude proved are precisely "ordinary open-source projects": cJSON, libexpat, decompression libraries, image decoders — things you use every day that nobody would spend twenty person-years proving. Anthropic's argument: when proof-writing drops from "expert-years" to "a token bill," formal verification can graduate from "crown jewel" to "routine tooling" — the way static types and unit tests once went from optional to standard.

And costs are still falling. In a controlled experiment on 36 small verification tasks, Opus 5.5 proved half the tasks using about 60% of the tokens Opus 5 needed — a 40% saving. Proofs are also reusable: letting agents share a workspace and copy each other's work (much of every proof is duplicated C-runtime and ASan initialization code) cut token spend another 17% — and two tasks that solo agents couldn't complete got done by copying. It mirrors how Mathlib, Lean's mathematics library, grew: prove the foundations once, and everyone builds on them.

The honest caveat: "verified" does not mean "bug-free"

Anthropic devotes a full section to limitations, and that honesty deserves credit — read it before getting excited. The core sentence: "Formally verified" does not mean "bug-free."

The guarantee has strict boundaries. It assumes the program is single-threaded, statically linked, with a fixed memory budget, running on a RISC-V CPU under Linux. In the real world, these libraries overwhelmingly run on x86 or ARM, dynamically linked against glibc. Proven crash-free on RISC-V doesn't imply crash-free on the x86 build on your server — instruction-set semantics and libc implementations differ in between.

The proofs also cover only "memory safety plus no failed assertions." Memory-safety issues ASan can't detect, business-logic errors (wrong answer, no crash) — all outside the guarantee. And not every entry point of the 123 libraries was proved — 239 fuzz entry points were.

But Anthropic's response is elegant: these limitations tell you exactly where remaining bugs could hide — either inside the four trusted components (RISC-V/Linux semantics, theorem statements, checker) or outside the proofs' scope. That collapses an unbounded problem ("bugs could be anywhere") into a bounded one ("harden four components"). Hardening the trusted base is amortized across every project that uses the proofs — prove once, the whole ecosystem benefits.

The byproduct worth more than the product: hundreds of bugs trillions of test runs missed

The most dramatic part of this research isn't the 180 million lines of proof. It's the hundreds of bugs found along the way while writing them.

The logic: when Claude got stuck writing a proof, it often meant the code genuinely had a bug — because "prove no input crashes this" is unprovable for buggy code. Anthropic let Claude patch the bugs it found (regenerating the spec and re-proving afterward), and the haul included things fuzzing had missed despite trillions of runs:

• A counter underflow in a decompressor's copy of the code, causing an out-of-bounds write — but only on ~20 MB inputs with one rare, specific layout. Fuzzers cap inputs around 1 MB and follow code coverage; they never reach that corner.
• An out-of-bounds read in an image decoder needing a 64 MB input and over 2 GB of memory. Beyond any fuzzer's resource budget.
• CVE-2026-56132 (libexpat): requiring deeply nested input — rediscovered by Claude.
• And a nastier category: logic bugs that never crash at all. In one library, a security feature restricting which files an application may load turned out not always to be enforced — invisible to fuzzing forever, since nothing crashes.

Two side experiments drive the point home: Anthropic had Claude write a formal spec of RFC 1950 and prove the Linux kernel's zlib implementation against it (at x86 binary level), surfacing minor corner-case bugs now being filed upstream; and in a verification attempt on a widely used PDF reader, one week surfaced as many security bugs as a typical year's reports. A logic bug latent for over a decade was also caught.

The subtext is worth savoring: fuzzing is sampling; proof is a census. Sampling, however diligent, has corners it never reaches. A census leaves no corner unexamined. Anthropic's numbers are the first hard quantification of that gap: trillions of test runs versus one proof.

Drawing the boundary with the OpenSSH shrink-wrap story: two security paradigms

Remember our cycle87 piece on Claude "shrink-wrapping" OpenSSH and 24 other open-source projects? That was Anthropic having Claude delete unneeded code — attack-surface reduction. This one is formal verification. Both are "Claude does open-source security," but the paradigms differ completely. One sentence draws the boundary:

Shrink-wrapping subtracts: remove code where bugs could hide, leaving attackers nowhere to stand. This work guarantees: keep the code, but prove mathematically it cannot crash.

The first philosophy is "less code is safer" — delete unused protocols, disable on-by-default features, and bugs disappear with the code. The second is "you don't need to delete, as long as you can prove." One shrinks the battlefield from outside; the other establishes order from within.

The interesting part is that in Anthropic's hands these are a combination play: shrink first (less code to prove), then verify (cheaper proofs on the remaining core). The blog doesn't spell it out, but the logic composes — the less code there is to prove, the cheaper the proof. For open-source maintainers, the realistic path may be: slim the project with shrink-wrapping, then buy "mathematical insurance" for the critical paths with formal verification.

Why "proof" fits open source better than "testing": the power law of dependencies

There's a strategic layer many will miss, hidden in one Anthropic sentence: "Dependencies follow a power law: proving even a small set of shared libraries makes a huge swath of software feasible to prove."

Consider how many open-source libraries your app depends on: dozens direct, hundreds transitive. Each author "tested" their library — but testing doesn't compose. Library A tested plus library B tested does not equal A+B tested. Combinatorial explosion makes "test everything together" forever impossible. That's the truth of the modern software supply chain: we stand on a tower of hundreds of "probably fine" libraries, fingers crossed.

Proofs compose in a way tests don't. Once a library is proven memory-safe, every downstream consumer inherits that guarantee without re-proving. And dependencies follow a power law: glibc, OpenSSL, zlib, libexpat — a handful of "bedrock" libraries underpin millions of projects. Theorem's roadmap (verify the entire Nix ecosystem, starting with glibc, OpenSSL, and cURL) bets exactly on this: prove the bedrock first, then build upward layer by layer. Same growth logic as Mathlib: prove the foundations once, everyone cites them.

So don't just read "123 libraries" as "many" — read it as "well-chosen." They're all high-frequency foundational libraries from OSS-Fuzz, where proof ROI is highest: one proof, hundreds of thousands of downstream beneficiaries. That's also why Anthropic says verification will go from bolt-on to built-in: someday your package manager may display a "verified" badge next to downloads and license, and you'll check it the way you check those today.

The first tilt of the offense-defense balance: a new weapon for defenders

Back to that opening line — "AI can now easily find and exploit bugs." For two years, AI in security has favored offense: vulnerability-finding agents keep getting stronger while humans still patch by hand. Last week's PHP2Shell episode was the pattern: AI helped find the flaw, humans pulled the all-nighter. The AI leverage has been asymmetric.

Formal verification is the first asymmetric weapon for defenders. The attacker's edge is "you only need one bug"; the defender's curse is "you must close every bug" — and proof is precisely the mathematical tool for closing every bug of a class. When Claude proves "this entry point cannot crash on any input," fuzzing on the attacker's side becomes obsolete at that layer: not "harder to find bugs" but "this bug class mathematically retired."

The arms race won't stop, of course. Anthropic concedes the proofs cover memory safety and assertions only — logic flaws, misconfiguration, supply-chain poisoning are out of scope. Attackers will move where proofs don't reach. But that's the shape of progress: use mathematics to mass-eliminate the low-level errors, forcing both sides onto harder problems. Like memory-safe languages (Rust) retiring whole bug classes and pushing vuln research toward logic flaws — not the destination, but the waterline rising.

For indie developers the implication is concrete: every open-source dependency you pull in may soon carry a "proven" badge. You won't need to know Rocq or write specs — you'll just read the badge, the way you read weekly downloads and last-commit date today. The democratization of formal verification may ultimately look that plain: one more glance before npm install.

A thought experiment: the code you didn't write

Here's a way to feel what this changes. Pick any project you shipped with AI assistance this year. Somewhere in its dependency tree sits a C library written before you were born — a PNG decoder, a regex engine, a TLS handshake. You've never read its source. Your AI coding assistant never read it either; it just imported it. That library has been fuzzed for years, has a CVE history, and still ships in billions of devices. Your entire product's security rests, in part, on code nobody on your team understands.

That has always been the deal with open source: trust the commons. Formal verification changes the quality of that trust. "This library survived a decade of fuzzing" is a statement about the past. "This library's entry points are proven crash-free for all inputs under 64 MB" is a statement about the future — about inputs nobody has tried yet, including inputs an attacker will craft next year. The first is a weather report; the second is climate physics.

And here's the part that should make every vibe coder pay attention: the libraries being proven are exactly the ones your AI-generated apps silently depend on. The agent that writes your code stands on the shoulders of cJSON and libexpat. Now another agent is checking whether those shoulders are solid. The stack is becoming self-certifying, layer by layer, from the bottom up. You may never write a line of Rocq in your life — but you'll ship on top of proofs the way you currently ship on top of tests you didn't write either.

What it means for developers in practice: the free pipeline is already open

This research isn't academic muscle-flexing; its productization moves are concrete:

First, Theorem's OSS Verify program runs this pipeline for qualifying open-source projects for free. Enrollment is by pull request (adding project.yaml, Dockerfile, and build.sh); Theorem reviews along lines similar to OSS-Fuzz, prioritizing widely used critical infrastructure. Maintainers receive a Rocq-kernel-checked proof that each fuzz target can't crash on any input up to 64 MiB; bugs found during the effort are disclosed privately by email with no disclosure deadline. And the roadmap is aggressive: verifying the entirety of the Nix ecosystem, including glibc, OpenSSL, and cURL — the bedrock of the modern software stack.

Second, a Claude skill is published to help developers verify their own code. Anthropic's closing pitch is direct: "We encourage all developers to explore formal verification with LLMs — especially those who previously tried it with less capable models and found it didn't work."

Third, the proofs and semantics are open. A subset of the proofs, the RISC-V and OS semantics, and specs for fuzz targets that couldn't be proved are on GitHub. Anyone can verify independently — the advantage of "proof" over "paper": you don't need to trust Anthropic's argument, you just run the checker.

For vibe-coding readers there's one more resonant layer: AI-written code is now being proven correct by AI. We worry "does AI-generated code have bugs" — Anthropic's answer is to have another, more rigorous AI process prove it. The code-writing agent and the proof-writing agent, one optimized for speed, one for correctness — possibly the standard division of labor in future software engineering.

Closing: certainty finally gets a price tag

In the history of software security, "certainty" has always been a luxury good. Testing gives you "probably fine." Fuzzing gives you "ran it a lot, nothing broke." Only formal verification gives you "mathematically impossible to break" — at a price only Airbus and the military could pay.

What Anthropic did is put the first affordable price tag on certainty: a few weeks, a token bill, 123 libraries. It's imperfect (RISC-V assumptions, memory-safety only, partial entry-point coverage), but it walked the road end to end: from "hand-proofs by experts" to "LLM proofs at scale," from "crown jewel" to "standard open infrastructure."

Anthropic closes by predicting verification will mature from "bolted-on" to "built-in," like static types and test frameworks before it. The subtext: tomorrow's CI pipelines will run "tests" and "proofs" side by side — the former saying "this change didn't break known behavior," the latter "this change cannot introduce this class of bug." When proofs cost what fuzzing costs, projects that skip them will look the way test-less projects look today: unprofessional. The question for maintainers will shift from "can we afford to verify?" to "can we afford not to" — and for the critical libraries at the bottom of the dependency tree, the answer is already becoming obvious.

From "finding bugs" to "proving their absence" — the gap isn't just technical. It's an upgrade of the entire security worldview. And this time, AI pressed the upgrade button itself.

Primary source: Anthropic's official research blog, "LLMs are making formal software verification feasible at a much larger scale," published 2026-10-22 (page metadata: Published: 2026-10-22T00:00:00.000Z, Updated: 2026-10-23T17:10:27.000Z). Cross-verified by Unite.AI's independent same-day coverage, "Anthropic Says Claude Proved Security Properties Across 123 Libraries" (2026-10-22), plus same-day analysis from siliconnoon and AI Weekly. Cover: macro photograph of a chip die (Hackaday RISC-V project) — a nod to the "assembly instruction level" at the heart of this work.