Great at Assembling Molecules, Weak at Picking Drugs: OpenAI's MedChemBench Grades AI's Medicinal Chemistry
On 20 November 2026, OpenAI released MedChemBench: 99 medicinal-chemistry problems from 117 papers and patents, covering 761 molecules. GPT-6.2 Astra scores 84.5% overall — but 95.4% on assembling structures and only 50.4% on choosing compounds. The total flatters; the components tell the truth.

In short: 84.5% overall, but only about half on the two things that matter most
On 20 November 2026, OpenAI released MedChemBench, a new benchmark for medicinal-chemistry reasoning: 99 problems drawn from 117 papers and patents, covering 761 distinct molecular structures. The headline number looks strong. At the highest "xhigh" reasoning setting, GPT-6.2 Astra earned 84.5% of the available credit, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0%, and GPT-5.6 Sol 58.2%. Read on its own, that looks like a story about AI learning to do drug research.
But OpenAI split the total open in the same announcement, and the story changes once you see the parts. GPT-6 Astra scored 95.4% on reconstructing molecular structures and 91.05% on explaining experimental evidence — but only 50.42% on selecting which compound to pursue next and 47.02% on predicting numerical properties. The model is best at the two things furthest from real decisions, and weakest at the two things medicinal chemists actually do every day: choosing the next compound to synthesize and test. This benchmark's value is not the 84.5%. It is the first time "assembling molecules" and "picking drugs" have been scored separately.
Medicinal chemistry is the most expensive, most time-consuming stage of drug discovery: a candidate molecule has to satisfy a stack of competing requirements at once — potent against its target, absorbable by the body, not broken down too fast, not too toxic. Fix one property and another can get worse. A chemist's job is to make decisions inside those trade-offs: design a batch of related molecules, make them, test them, then decide what to try next. MedChemBench tests exactly that process. The model is handed everything known halfway through a research program — the target, the goals, the compounds tried so far, the experimental results — and must answer: what should we pick next? If that ability genuinely holds up, AI can enter the laboratory's decision chain. If it does not, the model is still a very well-read bystander.
What it tests: four abilities, seven categories
MedChemBench splits medicinal chemistry into four abilities, scored separately. First, molecular structures: assemble the molecule correctly from the drawings and chemical groups provided, with atom connections and stereochemistry right. Second, scientific reasoning: explain results with the evidence supplied, such as why one compound was chosen over another. Third, ranking and selection: order compounds and pick candidates that meet the stated requirements. Fourth, property predictions: predicted numbers and categories must match the measured or calculated values reported in the source material, with partial credit for coming close.
The 99 problems fall into seven categories: integrated lead selection with 48 tasks, structure-activity relationship forecasting with 26, and smaller groups covering ADME (absorption, distribution, metabolism, and excretion) and delivery, molecular design and enumeration, safety and liabilities, retrosynthesis, and assay biology and mechanism. Eighty-five tasks span more than one scientific-content domain — a single problem can depend simultaneously on potency, selectivity, cell assays, metabolism, exposure, and safety, which is what a real lead-selection decision looks like. Every problem stops at a decision point in a research program: the model sees what was known at the time, and later results are withheld to build the reference answers and grading criteria.
The announcement's worked example shows the benchmark's temperament. One problem draws on research into inhibitors of PRMT5, a protein studied in cancer drug discovery. The model must predict how three related compounds will behave: how strongly they block PRMT5 in cells, how readily they cross a cell barrier, and how quickly liver enzymes break them down in a laboratory assay. The withheld results reveal a classic trade-off: compound 23 blocked PRMT5 in cells at a lower concentration — more potent — while compound 24 crossed the cell barrier more readily and broke down more slowly in the liver-enzyme assay. There is no all-round winner. The model has to weigh three properties at once rather than topping any single one.
Another problem tests something more basic: spatial reasoning. The model must assemble a molecule from a central structure and two attached groups while preserving an 18-atom ring. At the highest reasoning setting shared by all eleven models, GPT-6 Astra reconstructed the compound correctly in all ten responses. GPT-5.6 Sol failed all ten, drawing a 15-atom ring; GPT-5.6 Luna failed all ten, splitting the molecule into two disconnected fragments. Get the ring wrong and you are analyzing a different molecule — however elegant the property reasoning that follows, it starts from the wrong place. That example is the benchmark's methodology in miniature: first confirm the model identified the molecule, then talk about whether it understands the chemistry.
The scorecard: great at assembling molecules, weak at picking drugs
Totals first. At xhigh, GPT-6.2 Astra scored 84.5%, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0%, GPT-6.1 Terra 75.1%, and GPT-5.6 Sol 58.2%. GPT-6 Astra reached 83.8% at maximum effort. Higher reasoning settings improved scores, while large differences between models remained — a 26-point gap separates GPT-5.6 Sol from GPT-6.2 Astra. The evaluation is not small: 11 GPT models, 61 model-and-effort combinations, 60,390 graded responses, with ten retained responses for each task in every condition.
Then the components, which are the point. GPT-6 Astra at xhigh scored 95.37% on molecular structure, 91.05% on scientific rationale, 50.42% on ranking and candidate selection, and 47.02% on numerical prediction. And the four components do not weigh equally: structure accounts for 37.49% of available credit, rationale for 35.78%, ranking and selection for 17.51%, and numerical or categorical prediction for 8.92%. More than 70% of the total's weight comes from the model's two strongest areas; the two areas that best represent "making decisions" together account for about a quarter. The 84.5% total is an average lifted by the easy components.
The 50.42% figure needs particular care. Ranking is scored by pairwise comparisons, where a uniformly random ordering has an expected score of around 50%. In other words, GPT-6 Astra's performance on "which compound next" is not far from chance level. The 47.02% on numerical prediction is similar — predictions earn partial credit for closeness to measured values, but the model earns less than half the credit on that dimension. OpenAI cautions in the announcement that the categories use different tasks and scoring rules, so the percentages are not directly comparable as measures of difficulty. But even on the most generous reading, the conclusion does not change: between "understanding" and "choosing", there is a gap the models have not crossed.
One timeline detail is easy to miss: the technical paper is dated 9 November, and its 11 tested models do not include GPT-6.2 Astra — the 84.5% result appears only in the 20 November announcement, not in the paper's evaluation tables. That is not fabrication, just chronology: the paper was written before the new model was run. But it means the headline number did not go through the paper's full 60,390-response evaluation pipeline; it is a result added separately in the announcement. Keep that in mind when reading the total.
How the 84.5% is calculated: the scoring rules are half the story
MedChemBench's grading is a hybrid of deterministic checks and model-judged rubrics, not a single model marking its own homework. Molecular structures are checked deterministically with RDKit: the SMILES text the model submits — a text representation of a molecular graph — is converted to canonical isomeric form, preserving stereochemistry and isotopes, then compared against the reference structure. Get one atom connection wrong and it does not count. Numerical predictions generally receive partial credit based on fold error: the closer to the measured value, the more credit. Scientific rationale is scored by a separate GPT-6 Astra judge at medium reasoning effort, awarding zero, half, or full credit against task-specific criteria. Rankings are scored by pairwise comparisons; selection problems reward overlap with a reference set.
The advantage of this mixed grading is honesty: it does not pretend every ability can be measured with the same ruler. The price is that the total becomes a weighted average, and the weights themselves are a judgement — structure and rationale dominate because they appear in every problem and are easy to quantify; ranking and prediction weigh less partly because they apply only to subsets of tasks. Reading the weight table before reading the total is the first lesson this benchmark teaches.
One design choice deserves attention: every answer was a single attempt, with no tools, no feedback, and no new experiments. The model cannot look up a database, calculate a property, or revise and resubmit. That is a deliberately bare-handed setup by today's agentic-evaluation standards. Its virtue is that what gets measured is genuinely the model's own reasoning; its cost is distance from real workflows, where a medicinal chemist looks things up, runs calculations, and talks to colleagues. OpenAI's proposed extensions aim squarely at that gap: prospective cases, expert adjudication of alternative answers, explicit treatment of assay uncertainty, and an agent-based track allowing retrieval, calculation, structure verification, and multistep revision.
Three red lines OpenAI drew itself
The most creditable part of this announcement is that OpenAI put its limitations in the main text rather than an appendix. The first red line: "These results do not establish whether models can discover new drugs or improve laboratory decisions." The benchmark measures reasoning from historical evidence, not real drug discovery — the problems come from published research, and the answers already exist in the literature.
The second red line is contamination: prior exposure of the models to the 117 source papers and patents "cannot be ruled out". The authors ran a heuristic check, adding source DOI or patent identifiers to all 99 prompts and re-running nine models at xhigh. Mean overall-score changes ranged from a 0.23-percentage-point decrease to a 0.80-point increase, and every model-specific 95% confidence interval included zero. The check is neutral by design: it neither proves contamination exists nor proves it does not — it only shows that adding the source identifiers did not systematically change scores.
The third red line concerns the benchmark's own lifespan. The paper describes MedChemBench as close to saturation in its current form — top models are already near the ceiling on structure and rationale — and says open-ended design and synthesis grading remains partly tied to reference answers, which may penalize plausible alternatives. In other words, this benchmark's job is not to hold the leaderboard for years but to separate "can assemble" from "can choose" first, and let the next generation of benchmarks test harder things: prospective cases, expert adjudication, and an agent-based track.
Why the benchmark is still worth taking seriously
With the three red lines drawn, what remains is clearer. First, it tests decisions, not knowledge. Earlier science benchmarks mostly asked whether a model "knows" — can it recall a molecular formula, can it recognize a reaction type. MedChemBench asks whether it "can choose" — weighing conflicting properties to make a call. That shift mirrors the shift in coding benchmarks this year: BuggyMLE does not just check whether the metric recovered but verifies each injected bug; MedChemBench does not just report a total but scores decision-making separately. Evaluation is moving from testing knowledge to testing judgement, and that is the most important trend in benchmark design this year.
Second, the component design leaves "lopsided" nowhere to hide. A benchmark that reports only a total rewards models that pile up points on the easy parts; one that reports four components separately forces everyone to admit the weak ones. GPT-6 Astra's 95.4% and 50.42% on the same table is itself a form of honesty — OpenAI could have published only the 84.5%, but it printed the 50.42% too. For readers, that carries more information than any press-release language.
Third, it leaves a path for successors. The paper spells out its extensions: prospective cases (testing on unpublished research to close the contamination gap), expert adjudication (letting chemists judge whether alternative answers are reasonable), and an agent-based track (allowing retrieval, calculation, and multistep revision). A benchmark that writes its own succession plan on launch day shows unusual self-awareness — and that is part of why it is credible.
What it means for AI drug discovery
The timing is pointed. For two years, the AI drug-discovery story has been "models understand chemistry better and better": bigger models, more papers, prettier molecule-generation graphics. MedChemBench twists that story: understanding chemistry and making drugs are two different things. A model can assemble 761 molecular structures almost perfectly and still be a coin flip on "which one do we synthesize next" — and the latter is where laboratories actually spend money. Synthesizing one compound takes weeks; testing a round costs real money. Choosing wrong costs cash, not tokens.
For teams building AI drug-discovery tools, this benchmark is a new exam, and a hard one to game. The old way to score well was to make the model read more and recognize more structures. With the weights published, the room to pad the score is small: structure is already at 95.4%, leaving less than five points of headroom. The real gap is in ranking and prediction, and those are precisely the areas that "reading more papers" cannot fix — they require weighing conflicting evidence and judging uncertainty itself. MedChemBench pushes competition from "who read the most papers" toward "who makes the best decisions", and that is a signal for the whole field's research direction.
For investors and onlookers, the benchmark offers a way to judge claims without listening to vendor stories. The next time a team says "our model accelerates drug discovery", the first question can be: what are the four MedChemBench component scores? A total without components — or components where ranking and prediction still sit near half — means the model is at the same level as GPT-6 Astra: well-read, good at assembling, but not yet at the point of making decisions for chemists. The greatest value of a good benchmark is making exaggeration difficult. In that sense, MedChemBench has already done half its job.
Drawing the line with the GPT-6.2 launch article
This site published the GPT-6.2 Astra and Sol launch article on 17 November, anchored on the models themselves: stronger, cheaper, but with independent evaluation yet to catch up. This article is anchored on something entirely different: a benchmark released on 20 November, in which GPT-6.2 Astra appears only as a score — and one added in the announcement, outside the paper's evaluation tables. The topics, sources, and judgements differ; there is no duplication. Every mention of GPT-6.2 here is as a benchmark result, not as a model launch, pricing, or feature story.
For people building applications, the real lesson of this release is not about medicinal chemistry but about evaluation method. The easiest mistake when evaluating a model or an agent internally is to look at a single total: an 84.5% total could be an average of 95 and 50, or four components all near 84 — and those two situations mean completely different things for selection. MedChemBench's approach transfers directly: split ability into components, score each separately, publish the weights, and admit the weak spots. Once scores are split, choosing a model, writing prompts, and deciding "can this job go to the model" all have evidence behind them.
Three judgements
First, the 84.5% is a real result, but read it by component. GPT-6.2 Astra's total lead comes mainly from structure and rationale; on ranking and selection and numerical prediction, even the best models score only about half the available credit. Any report that writes MedChemBench up as "AI can do drug discovery now" is misreading the benchmark.
Second, only OpenAI's own models have been tested so far. All eleven models are GPT models, with no outside model in the comparison and no external calibration of the scores. GPT-6.2 Astra's 84.5% beats GPT-6 Astra's 83.3% by 1.2 points; on a 99-problem benchmark, the paper offers no direct significance judgement for that gap. The ranking gains a real reference point only when third-party models run the same test.
Third, clinical development is outside this benchmark's range. MedChemBench covers preclinical drug discovery; none of its seven categories touches clinical trials, dosing, or real patient data. A high score here does not mean a model belongs in clinical decision-making. OpenAI's red line is clear: this is a test of reasoning over historical evidence, not proof of drug-discovery ability. The most important thing when reading this release is not to confuse the two.
Three things are worth watching next: whether any non-OpenAI model runs the same test and supplies the missing external calibration; whether the proposed agent-based track materializes — and whether ranking and prediction climb from around 50% once retrieval, calculation, and multistep revision are allowed; and whether prospective cases can close the contamination gap by testing on unpublished research. Any one of those would rewrite the MedChemBench story.
Comments (0)
Sources
Related articles
On 18 November 2026, Google Research and co-authors posted BuggyMLE to arXiv: bugs injected into 222 working ML repositories, and 14 models slide from 95% on the easiest tier to 9% on the hardest. A 9-billion-parameter model trained easy-to-hard beats Claude Opus 5.5 on the 10-bug tier.
On 20 November 2026, Thinking Machines Lab released Inkling 2 Mini: a 26.4B-parameter looped transformer that runs its middle 12 layers three times per token, fits in about 20GB, and runs offline on a laptop. It leads open-weights models of its size on DeepSWE and SWE-bench Verified — with the caveats printed in the same announcement.
On 17 November 2026, Microsoft launched MAI Code 2 and Code 2 Flash: the larger model has 77 billion active parameters and a one-million-token context, averaging 63.2% across four benchmarks — close to GPT-6.1 Sol and Claude Opus 5.5 — at $0.309 per task. Read the three caveats first: promotional pricing, the lowest score of the three, and no long-horizon work.