Anthropic Hands Outside Reviewers a Badge: METR Can Publish Risk Findings Without Approval

On October 15, Anthropic announced it was effectively "hiring" a six-person team from the nonprofit evaluator METR (Model Evaluation and Threat Research): company-issued hardware, office desks, and access to internal tools and systems — a setup roughly equivalent to Anthropic's own internal risk-assessment teams. But the badge isn't the story here. The publication rights are: METR can publish its assessments of Claude model risks, incidents, and safety practices without asking Anthropic for permission. All Anthropic can do is demand redactions within narrowly defined categories — customer data, legally privileged information, model architecture details — and METR has the right to publicly question whether those redactions are justified.

The drama of the announcement lies in its prequel: two days earlier, Reuters exclusively reported that as recently as October 5, Anthropic and OpenAI were still seeking substantive control over what outside evaluators could publish, worried that evaluation reports might leak trade secrets. Ten days, from "we want approval rights" to "publish freely" — the most interesting U-turn in AI safety governance this year.

1. More than keys: handing over the "publish button"

What Anthropic unveiled is not an ordinary partnership but a three-tier "embedded evaluation" framework, with access and publication rights graded along two separate axes:

Tier one is represented by the biosecurity nonprofit SecureBio, handling narrowly scoped, targeted reviews — it is currently reviewing the chemical and biological weapons chapter of Anthropic's August 2026 risk report. This isn't its first time grading Anthropic's homework: in February it evaluated the previous risk report, judging the risks tied to Claude Opus 4.6 as "very low but not negligible" for existing threats and "low but uncertain" for novel ones. Tier-one publication requires company approval — the most conservative tier in the framework.

Tier two is Accenture's Faculty unit, covering broader model evaluations and adversarial testing. It gets more information and can publish without company approval, subject to narrowly defined redactions — which the evaluators themselves can publicly challenge.

Tier three is the star of this announcement, METR: six researchers with employee-level operational access, working directly with company hardware and internal systems, and publication rights identical to tier two — disclosure by default, no approval needed. Anthropic stated explicitly that it cannot veto METR's findings on risks, incidents, and safety practices. Three categories sit outside the open scope: customer data, legally privileged information, and model architecture details.

Anthropic also disclosed that it is in talks with Hugging Face and others about new evaluator partnerships — unconfirmed, and still at an early stage.

It's worth noting that the tiering itself is a response to months of controversy: when Anthropic named Faculty its first embedded evaluator in September, access levels and reporting mechanisms were "still being worked out." A month later, the answer arrived as a three-tier framework. The speed — from vague promise to a comparable yardstick — suggests the lab has realized something: on independent evaluation, the cost of staying vague now exceeds the cost of taking a clear position.

Illustration of Anthropic's three-tier evaluator framework and the timeline of its shift from publication control to default disclosure

Figure 1: Anthropic's three-tier external evaluator framework. Access depth and publication independence are two separate axes: tier three (METR) has the deepest access, while tiers two and three share the same publication freedom — disclosure by default, with redactions open to public challenge. Scores are illustrative.

2. The ten-day U-turn: October 5 → October 8 → October 15

Straighten out the timeline and the speed of the reversal is startling. According to Reuters' October 13 exclusive, two people familiar with the matter said that as recently as October 5, Anthropic and OpenAI were seeking "strict editorial control" over what evaluators could publish; by October 8, another person said talks with Anthropic had "evolved" toward openness. The exclusive hit on October 13; the three-tier framework was announced on October 15.

One detail deserves savoring: Anthropic CEO Dario Amodei had said publicly just last month that external reviewers should have the right to publish key findings "without editorial control," with OpenAI voicing support for assessors' "editorial independence" days later. Fine words — while at the negotiating table the labs were still demanding substantive control. The gap between rhetoric and negotiating posture shows that "independent evaluation" remained a slogan for months, only forced into reality over these ten days.

Responding to Reuters' questions, Anthropic said that seeking significant control "did not accurately characterize all of its discussions" with independent evaluators. OpenAI said it was not seeking approval rights over the full text of evaluator reports, describing its standard practice as redacting confidential information while letting evaluators disclose whether major redactions were made. Both are drawing lines — but clearly not in the same place.

Illustration of Anthropic's three-tier evaluator framework and the timeline of its shift from publication control to default disclosure

Figure 2: From "controlling publication" to "default disclosure" — a timeline of Anthropic's shifting stance on evaluator publication rights (September–October 2026). Red dots mark key turning points; facts sourced from Reuters' October 13 exclusive and October 15 follow-up.

3. Take: the U-turn wasn't a change of heart — it was three converging pressures

Don't romanticize this as a triumph of "safety-first" values. It looks more like a calculated damage-control move, driven by three pressures tightening at once.

The first is the cost of exposure. Reuters' exclusive put the negotiating demands on the table: a CEO publicly calling for "no editorial control" while negotiators demanded "strict editorial control." Once that contrast was in print, the reputational cost of continuing to demand control outweighed the confidentiality benefit. A source leaking that talks had "become more open" on October 8 — five days before the exclusive ran — is hard to read as coincidence.

The second is pushback from the evaluator ecosystem. Last month, more than 200 researchers signed a public letter, Geoffrey Hinton among them, demanding that evaluators independently control their findings, get broad access to AI labs, and be protected from retaliation. When the most authoritative voices in the field draw a collective line, labs can no longer use old-style contracts — "we grant access, we keep approval" — to squeeze evaluators. Top assessors are not short of options.

The third is political. In late September, the White House Accord on Super Intelligence listed independent evaluation as one of four safety measures. The accord is only morally binding, but it turned "independent evaluation" into political correctness every lab must endorse. And Anthropic is precisely the company that made independent evaluation its brand: Faculty was named its first embedded evaluator last month, its CEO had just invoked "no editorial control" — getting caught working the opposite angle at the negotiating table would have turned the brand into a shackle.

So my read: this U-turn was rational, not revelatory. Anthropic did the math — at a moment when independent evaluation is backed by a White House accord, a researchers' letter, and media spotlights, being the first to genuinely hand over publication rights pays far more than clinging to control. That doesn't diminish the outcome, but seeing the motive clearly gives us cooler expectations for what comes next.

4. Take: who watches the watchers? The independence problem with self-issued badges

The prettier the framework, the sharper this fundamental question gets: the labs choose their own reviewers. Reuters made the point in both reports — experts explicitly worry that Anthropic and OpenAI decide who carries out the reviews. Rumman Chowdhury, former US Science Envoy for AI and CEO of Humane Intelligence, put it more bluntly: evaluators know this privileged access "comes at a cost."

For METR specifically, independence deserves a further discount: it is no stranger to Anthropic, having previously reviewed the company's risk reports. Familiar faces make for gentle supervision — reviewers know that an overly sharp report could affect access in the next round of cooperation. This isn't an accusation against METR's professionalism; it's a structural dilemma. The supervised party is also the "client," controlling the access that evaluators depend on most.

The redaction clause is the second gray zone. Anthropic's three carve-outs (customer data, legally privileged information, model architecture) sound reasonable, but Reuters' exclusive nailed the key point: what a lab considers crucial intellectual property often overlaps with the information needed to judge a model's risks. That overlap is where future disputes will concentrate. The "evaluators can publicly challenge redactions" design is clever — it turns the dispute itself into part of the transparency — but a challenge is still just a challenge: once information is redacted, the public never sees it.

One easily missed reservation: Anthropic said that for newly partnered evaluator firms, while the working relationship is being established, the company reserves the right to "comment on and approve" reports. Translation: the "default disclosure" of the three-tier framework has so far been delivered to exactly one party — METR. Newcomers serve a probationary period first. That means the framework's openness still depends, in the short term, on Anthropic's case-by-case judgment of each partner — the true grade of "independence" can only be verified when a second and third evaluator reach tier three.

5. Take: the demonstration effect is real — but the framework is voluntary

Reservations aside, Anthropic's real contribution may be this: it turned "independent evaluation" from a slogan into a comparable yardstick. Access depth × publication freedom — with those two axes on the table, any lab talking about "independent review" can now be asked: which tier are you offering? Does publication need your approval? Can redactions be challenged? That comparability is itself pressure on the whole industry.

By contrast, OpenAI's current stance looks more open than Anthropic's old position but more conservative than the new framework: no approval rights over full report text, redacting confidential information as standard practice, letting evaluators disclose major redactions. The two labs are not walking the same path — which is precisely the sign that the industry is entering a standards-setting window. Whoever puts a workable framework on the table first defines the passing grade for the word "independent."

For other frontier labs, the demonstration effect cuts both ways. On one hand, Anthropic raised the bar: anyone using "we have external reviewers" as a shield will now have to answer which tier they granted. On the other, Anthropic proved that "handing over the keys plus publication rights" is operationally feasible — trade secrets didn't automatically leak the moment evaluators walked in, and the "narrow redaction plus public challenge" design offers a middle road. Whether that middle road gets copied depends on whether the next lab is willing to accept the condition that "redactions can be challenged."

But don't applaud too quickly. The entire framework is voluntary: Anthropic itself says it "hopes every frontier lab embeds evaluators," and the White House accord is only "morally binding." Voluntary means no legislation, no enforcement — it can be walked back when the winds change. More concretely, as of the announcement date, METR has not published a single assessment based on its new access. The framework's value can only be verified when the first report that makes Anthropic uncomfortable sees daylight. Then we'll watch two things: how the redaction clause gets used under dispute, and whether METR dares to publicly challenge it.

One more open thread is Hugging Face. Anthropic confirmed talks, but they're "unconfirmed." Given Hugging Face's role as the core platform of the open-source ecosystem, its entry into Anthropic as an evaluator would itself be a cross-verification between the "open" and "closed" camps — worth watching.

Conclusion

The badges are issued, the keys handed over, the publication rights released. For Anthropic, this is elegant posture management; for the industry, it's an experiment in writing standards into reality. But "independence" was never going to be proven by a single announcement — it will be proven by the first unfavorable report, the first redaction dispute, and the first lab that follows suit. The watchers have their badges on. Now let's see whether they dare use them.

Further reading

• OpenAI safety lead resignation: the personnel fault line in safety governance
• Anthropic's anti-distillation safeguards: a lab fortifying itself
• Anthropic's frontier red team: self-replicating IoT threat research