Back to Explore
NewsVibeFix editorialUpdated Nov 20, 2026

Microsoft Research's TokenEconomizer: let coding agents learn from past sessions and cut SWE-Bench costs 18–31%

Microsoft Research's TokenEconomizer learns from an agent's past sessions in a repository and cut SWE-Bench costs 18.4–30.9% in the authors' tests. The blog post is from 16 November; the paper is from 15 October — and the plugin only pays off after several sessions in the same repo.

A developer at a dual-monitor workstation in a dark room, both screens showing lines of code

One sentence: Microsoft Research wants your coding agent to learn from its own history

On 16 November 2026, Microsoft Research introduced TokenEconomizer on its official blog: an open-source plugin for a long-standing annoyance with coding agents. In the same repository, day after day, an agent starts from scratch — rediscovering facts it already found, loading tools it never uses, and falling into the same failure again. TokenEconomizer does not touch the agent or the model. It watches from the side: after each session, it reads the session log and turns what was learned into guidance that makes the next session cheaper. Across 277 real bug-fixing tasks (SWE-Bench), it cut mean cost by 18.4% to 30.9%, came out cheaper in all 30 repository-by-model combinations tested, and left task success rates broadly stable.

First, the dating, so nobody mistakes this for something it is not. The TokenEconomizer paper (arXiv 2610.22732) was published on 15 October, and third-party outlets covered it on 30–31 October. What is new on 16 November is Microsoft Research's official blog post: the first time the project itself has presented the plugin systematically, with the numbers, the mechanisms, and the limits. Every core figure in this article comes from that blog post; paper details are only used where the blog does not go into them, and are marked as such. In other words, the news here is "Microsoft Research officially presents this plugin", not "this plugin appeared today".

Why is this worth writing about? Because the billing problem with coding agents has shifted from "is it expensive" to "where is the waste". Once usage-based billing arrived, many developers discovered their agents were not spending money on doing the work, but on repeating it: every session resends the instructions, the tool descriptions, and the whole conversation history to the model. The longer you work in one repository, the worse the waste gets. TokenEconomizer is the first attempt to package "learning from experience" as a general plugin: no model swap, no agent rewrite, no prompt overhaul — just the repository's own history, used to save money.

Where the money goes: three numbers

Microsoft Research did something refreshingly honest before pitching the solution: it profiled Claude Code on 60 SWE-Bench tasks first, to see where the tokens actually went. Three kinds of waste kept recurring, each with a concrete number attached.

The first is rediscovering the same facts. Once a repository had accumulated at least five completed tasks, 30% of tool calls repeated an action from an earlier task in that same repository; only 7% matched an action from another repository's history. In other words, an agent's "experience" is highly specific to one repository — but every new session behaves as if it has never been there before, re-learning how tests run, where the config lives, and what the build command is. The more you work in one repository, the higher the share of rediscovery, and the bigger the waste.

The second is paying for tools that are never used. The agent received 23 tool descriptions on every turn, but 16 of them were never called. Those descriptions are billed per token, every turn. The more tools you have, the more stable the waste: you think you are paying for capability, but about seventy percent of the tool-description spend buys nothing.

The third is the most expensive: getting stuck on failures. The 17% of tasks that failed accounted for 32% of the spending. Once an agent falls into the same error, it retries round after round, tries a different angle, retries again — burning tokens each time, without the success rate rising to match. A failed task costs roughly twice as much as a successful one.

The blog also reports a control experiment: the team tested three popular open-source cost-saving plugins, each built around one fixed strategy. Each one saved money in some repositories but not in others. The conclusion is blunt: effective cost control has to adapt to the repository; no single strategy works everywhere. That is why TokenEconomizer is designed to learn from history rather than to apply fixed rules.

Three mechanisms, one per kind of waste

After each session, TokenEconomizer reads the session's log and updates what it knows about the repository, then acts on the next session through three mechanisms. It only uses interfaces the agent already provides. It does not modify the agent or the model.

The first mechanism is micro-skills, aimed at rediscovery. TokenEconomizer identifies recurring patterns across sessions and proposes one-line notes for them — "run the tests with this command in this repository", say, or "this directory is generated, do not edit it directly". But a note does not take effect immediately: it is only added to the next session's starting context after several sessions have confirmed the pattern, and it is retired if later evidence contradicts it. That "confirm over several sessions, retract on contradiction" design matters — it stops the agent from learning the wrong lesson from a single coincidence and then compounding the error.

The second mechanism is tool pruning, aimed at unused tools. TokenEconomizer removes tools that repository history suggests the agent is unlikely to use, while always retaining the core tools for running commands and editing files. Removed tools can be restored if needed. In effect, each session carries only the toolbox this repository actually uses, instead of the whole workshop. Given the earlier number — 16 of 23 tools never called — this mechanism targets the most certain waste of the three.

The third mechanism is failure-loop intervention, aimed at stuck agents. It watches for an agent hitting the same failure repeatedly, and a judge model decides whether the agent is truly stuck or still making progress. If it is stuck, the agent gets a short note suggesting a different approach. In the open-source plugin there is one more layer of protection: if the agent is still stuck after the note, TokenEconomizer alerts the developer rather than stopping the work itself. It intervenes, but it does not overreach: the final call stays with the human.

The three mechanisms are independent, and the blog says each one saves money on its own, with all three together saving the most. The reported savings also include the cost of TokenEconomizer's own model calls — a point worth crediting, since many cost-saving schemes quietly leave their own overhead out of the headline number.

TokenEconomizer's three kinds of waste and three mechanisms: 30% of tool calls rediscover past actions, 16 of 23 tools are never called, 17% of failed tasks account for 32% of spend, addressed by micro-skills, tool pruning, and failure-loop intervention

How well does it actually work: numbers and limits together

The evaluation ran 277 SWE-Bench tasks from six repositories, across five agent-and-model combinations, with every repository starting from zero history. Mean cost fell by 18.4% to 30.9%, and every one of the 30 repository-by-combination cells came out cheaper. Pass@1 — the share of tasks solved on the first attempt — rose in four of the five combinations. The blog describes this both as "without performance loss" and as success rates remaining "broadly stable", and the two phrases need to be read together: overall nothing got worse, but individual combinations wobbled slightly. Third-party tabulations of the paper show one combination (Claude Code with Opus 4.8) slipping from 74% to 73% — while also posting the largest saving, 30.9%. The biggest saving and the small dip appearing in the same combination is not a coincidence; it is a trade-off. Something pruned away is occasionally something useful.

Another comparison is worth noting: simply turning down the model's reasoning effort saved a similar or even larger amount, but led to substantial performance degradation. That is exactly TokenEconomizer's selling point: it saves on waste, not on capability. Turning down reasoning effort makes the model dumber; TokenEconomizer makes it do less useless work. The bills can look similar; what gets delivered is not.

Savings grow as experience accumulates. Early tasks can cost more while TokenEconomizer learns, but on Claude Code, cumulative costs fell below the default in every repository by the sixth task. In other words, this is not something that pays off on day one — the longer you work in one repository, the better it gets. That also explains the results in two other settings: sequential web-development tasks came out 16.0% to 21.2% cheaper, while security auditing — where every task is in a new repository, so every run starts from zero — still saved 6.4% to 19.6%. A new repository saves something too, just much less, which is consistent with a design whose experience comes from repository history.

But the limits need stating plainly. SWE-Bench is real bug-fixing tasks on Python open-source projects; it is not your repository. 277 tasks, six repositories, five combinations is a respectable sample, but it proves "it saves under these conditions", not "it saves 30% in every repository". The blog itself says early tasks can cost more, and that cumulative savings only show up by the sixth task. If you do one or two sessions per repository and then move on, this plugin will not help you much. And author-run benchmarks are not independent replication: as of now, no independent third-party replication has been published.

One idea behind all three mechanisms

Look at the three mechanisms together and they turn out to be one idea in three forms: most of an agent's waste is not "not capable enough", it is "does not know". It does not know how tests run in this repository, so it rediscovers it every time. It does not know which tools go unused, so it loads all of them. It does not know it is stuck, so it keeps retrying. TokenEconomizer does not make the model smarter; it outsources "knowing" to the history log.

That puts it in a different direction from the usual cost-saving advice. Switching to a cheaper model, turning down reasoning effort, or shortening context all save money by using less capability, at the cost of worse results. TokenEconomizer saves money by doing less useless work, with capability untouched. That is why its success rate can stay broadly flat while turning down reasoning effort cannot. The difference is like saving electricity by turning off half the lights in an office where people are working, versus turning off the lights in rooms nobody is using. The bills may look similar; in the first case, someone is working in the dark.

Another design choice worth noticing is that it is a "black-box" controller. It does not open up the agent, change model weights, or touch agent code; it only works through interfaces the agent already exposes. The advantage is generality: the same plugin attaches to Claude Code, Codex, Copilot CLI, and Pi without a separate implementation for each. The cost is that what it can do is limited: it can only influence what the agent sees and what hints it receives, not change how the agent behaves directly. Its ceiling depends on how much interface the agent leaves open — and if an agent's interfaces change, the plugin's effect changes with them.

That black-box positioning also explains why the blog stresses that what it learns stays on the developer's machine. It reads session logs from your repository, which may contain your code, your test setup, and your failure history. Kept locally, none of that is uploaded to a server to "improve the product". For teams that care about code privacy, that matters as much as the money.

The blog also contains a comparison that is easy to miss: the authors first tried three existing open-source cost-saving plugins, each built around one fixed strategy. Each of them saved money in some repositories and not in others — no single fixed strategy worked everywhere. That result is the reason TokenEconomizer exists: how to save cannot be written down in advance; it has to follow the repository's history. Switching off an unused tool saves money in repository A and may switch off exactly the tool the next task needs in repository B. So the plugin is not doing "more aggressive saving"; it is doing "look at the history first, then decide where to save".

How to check whether it is actually saving you money

If you decide to try it, do not judge it on a single bill. The plugin itself says early tasks can cost more, and cumulative costs only fall below the default by the sixth task. So verify it like this: pick a repository you work in often, do six to ten tasks without the plugin first, and record the token usage of each; then install the plugin and do six to ten similar tasks in the same repository, and compare the averages. Keep the tasks as similar as you can — all bug fixes, or all small features — or the comparison means nothing.

When comparing, watch three things. First, look at the average and at the single most expensive task. If the average fell 20% but the most expensive task got more expensive, the plugin is backfiring somewhere, and that task's log is worth reading. Second, check whether the success rate moved. The blog's figure is Pass@1 broadly flat; yours should be broadly flat too. If success fell, the money saved was bought with quality, and it is not worth it. Third, count the plugin's own overhead. The blog says the reported figures include the controller's own model calls; your own check should include them too, not just the agent's bill.

There is also a simpler test: after ten tasks, ask yourself whether the agent has stopped asking you how the tests run. If that question has gone away, the plugin is working. If it is still being asked, either the micro-skills have not accumulated enough confirmations yet, or your repository changes so fast that history has little time to settle. The first is a matter of time; the second is a matter of fit — in a fast-changing repository, history is worth less by nature.

Is it worth installing: cheap to try, keep expectations modest

On adoption cost, the bar is low. It is open source (Apache 2.0), needs Python 3.10 or newer, no GPU, and no API key of its own; its model calls run through your own agent and login, and what it learns stays on your machine. It supports Claude Code, Codex, the GitHub Copilot CLI, and Pi, with experimental support for OpenCode, Cursor Agent, and a few others. Installation is cloning the repository, adding it as a marketplace, and running one install command. Trying it costs little more than installing a plugin and watching a few bills — nothing irreversible.

Two things to check first, though. One: the version ranges in the plugin's own documentation may already lag behind. Third-party write-ups list supported ranges such as Claude Code 2.1.300 to 2.1.310, while the latest release has already reached 2.1.325. Check the repository's README for your agent version before installing, rather than discovering an incompatibility afterwards. Two: it needs several sessions in the same repository before it helps. If you open one or two sessions per project and move on, it will not do much for you; if you spend every day in the same repository getting an agent to fix bugs and add features, it is built for you.

There is one more thing worth taking away, whether or not you install it: the three waste patterns can be handled by hand, without any plugin. Writing "how to run the tests, what the build command is" into your project's CLAUDE.md or AGENTS.md is a manual version of micro-skills; periodically cleaning up the agent's tool configuration and switching off MCP servers you do not use is manual tool pruning; giving the agent a rule like "after two failed attempts on the same error, stop and report" is manual failure-loop intervention. The plugin automates these things, but the ideas work without it.

Back to the headline numbers, then: 18.4% to 30.9% is not a promise that your bill drops by thirty percent. It is a range measured under the authors' conditions, starting from zero history, with cumulative savings showing up after the sixth task. Read it as "the waste in coding agents is real, and it can be measured", rather than "install a plugin and your bill drops by thirty percent". That is the real value of Microsoft Research's 16 November post: it did the accounting. Thirty percent of tool calls repeating the past, sixteen tools never called, seventeen percent of failed tasks eating thirty-two percent of the spend. The numbers themselves are worth remembering more than the plugin.

Views 4Comments 0

Comments (0)

ME
0/1000
Loading comments...

Sources

Browse projectsPublish your project

Related articles

A developer with glasses types on a laptop at a wooden desk, an external monitor to the left showing C code on a dark background, a small plant on the desk
News
Microsoft's Coding Model Stops Playing Second Fiddle: MAI Code 2 Scores 96% of Opus 5.5 for 5% of the Cost — Read the Three Caveats First

On 17 November 2026, Microsoft launched MAI Code 2 and Code 2 Flash: the larger model has 77 billion active parameters and a one-million-token context, averaging 63.2% across four benchmarks — close to GPT-6.1 Sol and Claude Opus 5.5 — at $0.309 per task. Read the three caveats first: promotional pricing, the lowest score of the three, and no long-horizon work.

Developer WorkflowProduct LaunchModel Updates
A developer types at a dual-monitor workstation; the left screen shows a dark code editor, the right a testing dashboard with green checkmarks and bar charts
News
Did Your Agent Get Better or Worse? AWS Open-Sources UAEF to Measure Agents from Every Framework with One Ruler

On 11 November 2026, AWS open-sourced UAEF: adapters turn traces from LangGraph, LangChain, Strands and Bedrock AgentCore into one format, then 36 metrics across 7 dimensions score them, flagging regressions past 5%. The judge model is hardened too — scores must cite verbatim evidence, and never decide safety.

Developer WorkflowProduct LaunchTesting & Quality
Against a black background, a conductor seen from behind raises a baton, with swirling trails of light circling beneath him like an orchestra being conducted
News
One API for 45 Models, and It Switches Providers When One Fails: Perplexity's Orchestrator Wants to Be the Switchboard

On 17 November 2026, Perplexity launched the Orchestrator API: one interface reaching OpenAI, Anthropic, Google, xAI and open-weight models such as GLM, Kimi and DeepSeek, billed at the answering model's published rate with no platform fee. When one provider fails, the same model retries on another deployment, and preview routing assigns requests to models by task.

Developer WorkflowProduct LaunchModel Updates