AI has entered the lab, but it can't staff it yet: OpenAI's BioBasicsBench leaves GPT-6 Astra at 66.7%
OpenAI's BioBasicsBench tests AI agents on 100 raw wet-lab tasks. Its best model, GPT-6 Astra, scored 66.7% — a big jump from GPT-5.6 Sol's 43.8% — but OpenAI itself warns the score does not yet mean reliable lab judgment.

On November 6, OpenAI released a new benchmark called BioBasicsBench. It does not test whether a model can recite a biology textbook. It tests whether a model can work the way a real bench scientist does — reading the raw evidence a laboratory produces every day: blurry gel photos, plates crowded with colonies, cells flickering in and out of view under a microscope, raw instrument files. One hundred tasks, all of them routine wet-lab analysis. It is the first time AI agents have been systematically put to the test on the lab bench itself.
The results are both encouraging and sobering. OpenAI's strongest model, GPT-6 Astra, averaged 66.7% at its highest reasoning setting, a large jump from GPT-5.6 Sol's 43.8% — and it used about half as many tokens to get there. But OpenAI's own conclusion in the announcement reads: "Higher benchmark scores do not yet mean models can make reliable judgments in the lab." That is not politeness. The second half of the post backs it up with three concrete failures.
This article covers three things: what the benchmark actually tests, how to read the 66.7% figure, and why a "good score" makes the mistakes worth taking more seriously, not less.
The verdict first: it tests a craft, not knowledge
Most existing science benchmarks are built around charts and images selected for publication — material that has already been cleaned up to tell a story. The model answers questions about a figure that has already been explained. A real laboratory is nothing like that. OpenAI makes the distinction explicit at the top of its announcement: everyday wet-lab work involves failed experiments, ambiguous images, conflicting observations, and a great deal of troubleshooting. Benchmarks built on polished figures cannot measure that ambiguity, or the skill of handling it.
BioBasicsBench was developed by OpenAI with Ligo, a biotech company focused on enzyme design. Its 100 tasks span five areas of routine lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Every task comes from data in Ligo scientists' own experiments, paired with expert-written grading rubrics, and each one went through three rounds of review: an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review. The reviewers checked the prompt, the experimental evidence, the rubric, the author notes, and a graded model attempt.
What the tasks hand the model is exactly what a bench scientist faces each day: gel and blot images (31 tasks), protocols, notes and specs (27), tables and instrument exports (20), bench, apparatus and reagent photos (18), microscopy (14), instrument screens and plots (13), sequencing files (12), protocol videos (7), agar plates and colonies (7), plus reference sheets as supporting material (21). A single task can combine several kinds of evidence. The model must return a structured answer — a conclusion, a measurement, or a recommendation, with supporting reasoning where required. Scoring awards credit against individual rubric criteria, so a correct conclusion with unsound reasoning earns a different score from a fully sound analysis.
The most notable design choice is that "recognizing when the evidence cannot support a conclusion" is itself a tested skill. For many tasks, the correct answer is not a number but an admission that the evidence is insufficient. That is the hardest hurdle for today's AI models: they are trained to always produce an answer, while science requires you to stay silent when the evidence is not enough.
What does 66.7% actually mean?
The units matter here, so let us be precise. OpenAI reports mean reward: the average task score, expressed as a percentage of the available credit, with partial credit allowed. 66.7% does not mean the model "solved 66.7 out of 100 tasks." It means it earned, on average, two-thirds of the available points. The difference is large: getting the conclusion right while the reasoning fails, or the number right while missing a key caveat, earns only partial credit.
OpenAI evaluated seven of its own models across 37 model-and-reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 graded results. At the xhigh setting — the highest reasoning-effort setting reported for all seven models — the scores were: GPT-6 Astra 66.7% (95% confidence interval 60.3–73.1%), GPT-6.1 Sol 62.0% (55.6–68.4%), GPT-5.6 Sol 43.8% (37.6–50.0%), GPT-6 Luna 38.3%, GPT-5.5 35.4%, GPT-5.4 33.2%, and GPT-5.2 31.2%.
Three things are worth pulling apart. First, the generational gap is real: Astra beats Sol by 22.9 percentage points, and their confidence intervals do not overlap (60.3–73.1% versus 37.6–50.0%), so sampling noise cannot explain it. Second, efficiency improved at the same time: Astra averaged 4,704 output tokens per task against Sol's 9,099 — a higher score for fewer tokens. OpenAI's own explanation is that stronger native multimodal capability may be the key, letting Astra pick up finer detail from experiment photos. Third, the ceiling is still far away: even the strongest model leaves a third of the points on the table, on 100 tasks that bench scientists "should be able to complete reliably."

One limitation must be stated plainly: all seven models are OpenAI's own. At launch there were no scores for Claude, Gemini, or any open-weight model, and no published human baseline. So the 66.7% figure can currently do only one thing — measure progress between two generations of OpenAI's own models. It cannot yet answer how far an "AI bench scientist" is from a human one, because the other end of the scale is blank.
Where it improved: three details done right
OpenAI gives three examples of Astra outperforming Sol, and each is worth unpacking, because they show judgment becoming more human, not just knowledge becoming broader.
The first is knowing what cannot be concluded. In a cell-imaging task, Sol classified cells as dead based on the signal from a dye used to detect dead cells. But the detector could also pick up light from a second dye, so brightness alone could not justify that conclusion. Astra recognized the ambiguity and called for better controls to separate the signals. This is not a victory of biological knowledge but of restraint — the model was not pushed along by something that merely looked like an answer.
The second is checking whether the control itself is sound. In a DNA cloning task, Sol saw no colonies on the negative control plate and based its conclusions on that observation. Astra noticed that the plate was labeled with the wrong antibiotic — kanamycin (LB Kan50), while the experiment used ampicillin — and correctly rejected it as a reliable control. The cover photo is the original evidence from that task: a gloved hand holding the plate, the handwritten "LB Kan50" on the rim circled. One mislabeled plate decides whether the whole experiment's conclusion stands. A model that catches that kind of detail is earning the right to talk about working in a lab.
The third is remembering the physical context behind the data. In a microscopy task, cells came into and out of focus across a series of images. Sol looked for a biological explanation for their apparent disappearance. Astra correctly recognized a z-stack — a series of images taken at different depths — where the cells "vanish" only because the focus moved. The model was not led astray by the surface phenomenon; it remembered how the data was produced.
Where it failed: three mistakes worth remembering
The most valuable part of the announcement is the three failure cases OpenAI shows without flinching. They say more about the true state of the "AI bench scientist" than the score does.
The first failure is letting a prior assumption override the controls. In a colony PCR task, Astra correctly distinguished the two groups of colony bands, but assumed that a larger PCR product would identify successful cloning, and proposed lanes 4, 6, 7 and 8. The reference interpretation, based on comparing against the controls, selects lanes 5, 9 and 10. The visual groups were right; the identities attached to them were reversed. The colonies Astra picked carry the old plasmid, so downstream purification would produce the wrong construct. OpenAI's wording is restrained, but the consequence is concrete: a correct intermediate observation failed to guide the final decision.
The second failure is generalizing from partial evidence. Astra concluded from a microscopy image that vacuoles — compartments inside cells — are generally darker than the surrounding cell. But some vacuoles in the same image appear lighter, because changing the microscope's focus can reverse the contrast. Astra inspected several regions and even commented that contrast varies with focus, yet its final answer kept only the darker description from the cells it called better focused, leaving the contradictory lighter regions out of the conclusion. Inspecting contradictory evidence and then not letting it into the conclusion is more worrying than not seeing it at all.
The third failure is the most ordinary, and the most telling: counting colonies wrong. On one plate, Astra placed a single counting mark between two neighboring colonies, left two visible colonies unmarked, and marked empty agar as a colony. The net result was an overcount of approximately 12% relative to the benchmark reference. Colony counts feed directly into estimates of how efficiently DNA was introduced into cells — with the plated volume and dilution held fixed, a 12% overcount inflates that estimate by the same proportion. This is not a failure of deep reasoning but of a basic operation that should be a glance-and-count job.
One hundred tasks, one map of abilities
The distribution of the 100 tasks is worth a look. According to the original table, protein methods account for 25 tasks and imaging and cytometry for 23 — the two largest groups — with the rest spread across quantification and assays, nucleic acid methods, and molecular cloning. More revealing are the skill tags attached to each task (one task can carry several): tacit knowledge 88 tasks, image and data reading 88, error detection 68, protocol understanding 50, and spotting insufficient data 19.
The center of gravity is clear: the benchmark tests "can you see, can you judge" far more than "do you know." Eighty-eight tasks demand tacit knowledge — the kind of experience that cannot be written into a textbook and only accumulates at the bench, like "that band does not look right" or "this color change means the reaction is not finished." Eighty-eight demand reading images and data. By contrast, "recognizing insufficient evidence" appears in only 19 tasks — the smallest category. That is a design tension worth noting: the very skill that most separates an "assistant" from a "bench scientist" carries the least weight in the test set. OpenAI does not explain the ratio, but for readers it means the 66.7% total contains only a small dose of "daring to say I don't know." If a future version raised that share to a third of the tasks, the score would probably fall — and it would be a more honest score.
One more detail: scoring awards credit against individual rubric criteria. A task's conclusion, numerical result, reasoning, and recognition of insufficient evidence each earn points. A model can get the conclusion right with faulty reasoning and earn half credit, or get the conclusion wrong while correctly flagging a problem with the control and still earn some credit. That design is closer to how a laboratory actually judges a scientist than binary right-or-wrong scoring: a bench scientist's value lies not only in whether the final number is right, but in whether they spotted problems along the way and left room for doubt.
Why now: from answering questions to doing work
The timing of BioBasicsBench says something on its own. For the past two years, the main battleground of AI evaluation has been "can it answer questions": mathematics, programming, professional exams, with models passing human-level performance on one clean test set after another. But answering questions and doing work are different things. In a test, someone else writes the question, the conditions are complete, and the answer is unique. At work, the evidence is messy, the conditions are incomplete, and the answer is often "run the experiment again." OpenAI's opening line — that everyday wet-lab work involves failed experiments, ambiguous images, and conflicting observations — is really a line drawn for the whole evaluation industry: from testing knowledge to testing judgment.
That shift matters commercially, too. AI coding assistants, AI support agents, and AI research assistants are entering real workflows today, and they face exactly the same conditions as the wet lab: dirty inputs, ambiguous instructions, and real costs for mistakes. A model that scores highly on clean test sets may walk into a real workflow and make confident errors — the way Astra sorted the PCR bands correctly and then picked the wrong colonies. BioBasicsBench is valuable because it turns that kind of "confident error" into something measurable, comparable, and improvable over time.
The boundaries are worth seeing as well. OpenAI writes at the end that tracking these capabilities will help it understand their implications for biosafety and biosecurity. The subtext is that a model that can read laboratory data is not far from one that can act on it. Reading a gel, recognizing a colony, understanding a protocol — these abilities are neutral in themselves, but they are also prerequisites for automating biological experiments. Today's discussion is still "can a model be a lab assistant"; tomorrow's may be "what kinds of experiments should a model be allowed to do, and who decides." The benchmark does not answer that question, but it makes the question concrete: when the score climbs from 66.7% to 85% or 90%, how far should the laboratory door open to AI? That is a question worth starting to think about now.
Why this announcement deserves a careful read, not just a glance at the score
Two judgments are worth stating clearly. First, the methodology of BioBasicsBench is more valuable than its scores. It demonstrates an evaluation approach: stop testing models on cleaned-up paper figures and test them on real, messy, raw evidence; stop scoring binary right-or-wrong and award partial credit against fine-grained rubrics; stop testing only "can it answer" and test "does it know when it cannot." That approach travels well beyond biology. Any field pushing AI into a real workflow — support tickets, financial reconciliation, code review — faces the same problem: a clean test set cannot measure dirty work.
Second, the announcement deserves the same reserve it asks of models. At publication, it contained no public link to the dataset, the rubrics, or the scoring code — an independent check on November 6 confirmed that absence. For now, nobody outside OpenAI can rerun the benchmark, reproduce the scores, or test whether the rubrics reward the right things. All the numbers come from OpenAI testing its own models on tasks built from its partner's experiments. That does not make the numbers untrustworthy; it makes them vendor-reported, pending independent verification. OpenAI also notes at the end that tracking these capabilities will help it understand their implications for biosafety and biosecurity — a sentence worth pausing on, because the gap between a model that can read lab data and one that can act on it is only an execution interface wide.
So the correct way to read 66.7% is this: the ability to read laboratory evidence is genuinely improving fast — from 43.8% to 66.7% in one generation, on fewer tokens — but it is still far from trustworthy. It catches a mislabeled control plate, and then picks the wrong colonies from a PCR gel. It recognizes a z-stack, and then overcounts a plate by 12%. A "bench scientist" who fumbles basic operations cannot yet work a shift alone.
For anyone building AI products, the real lesson sits in the final section: OpenAI says the goal is to help scientists spend less time troubleshooting and more time deciding which experiments to pursue next. That ordering is right — make "reading the evidence" reliable before talking about "running the experiment." And for every team pushing AI into professional workflows, this announcement is a prompt to check your own evaluations: does your test set contain dirty data? Does it include "the evidence is insufficient" as a correct answer? Is there a task designed specifically to catch the model being overconfident? If not, your score may be measuring the model's ability to recite, not its ability to work.