Back to Explore
NewsVibeFix editorialUpdated Nov 20, 2026

Amazon Retired Sonnet 4.6 From Ticket Labeling: A 4B Decision Model Took Over at 98% Less Cost

On Nov 9, 2026, Amazon engineers wrote that their internal ticket system swapped Claude Sonnet 4.6 for Strands Decider E4B: same labeling quality, cost per ticket from $0.0222 to about $0.0005. But the 98% figure has boundaries — and no accuracy number was published.

An engineer working in a data center aisle, dual monitors showing code

On November 9, 2026, two Amazon engineers, Carr and Mehul, told a refreshingly concrete story on the official Strands Agents blog: they had replaced the large language model in one of the company's internal ticket-labeling systems. The job used to belong to Claude Sonnet 4.6, a frontier model. Its replacement is Strands Decider, a 4-billion-parameter decision model that cannot write a single sentence. The result: classification quality "just as well," cost per ticket down from $0.0222 to about $0.0005 — a 98% reduction — and average time per call down from 4,215 milliseconds to 897 milliseconds, 78% faster. By the engineers' account, the swap will save "millions of dollars of token costs in a year."

This story is worth reading carefully, not because 98% is a startling number, but because it spells out something many teams are still vague about: not every job handed to a large model needs a large model. Labeling a ticket is, at its core, a multiple-choice question across 20 fixed categories, not an essay. Amazon's team found that when the task itself is "pick one from a known list," a small model that can only answer multiple-choice questions is not just cheaper and faster — in their head-to-head testing it actually performed better than Sonnet 4.6. The value of the post is not in its conclusion but in its process: it lays out every step, every number, and every pitfall, including a surprising one — the order of keys in a JSON request moved output probabilities by as much as 0.19.

How the old system worked

First, the background. This internal ticketing system is one of the largest inside Amazon. The tickets it handles include customer escalations, capacity requests, automated alarms, security findings, and anything else across the company that needs sorting — 20 category types in total. The old approach: an agent picked up each ticket, sent it to Claude Sonnet 4.6 with a long system prompt, and got back a JSON object containing a label and its reasoning. Sonnet 4.6 was, in the blog's own words, the "goldilocks choice" balancing intelligence with cost, and a nine-month-old model holding up at that scale is itself evidence it did the job well.

But "did the job well" and "was the only model that could do it" are different claims. The engineers' judgment was that ticket labeling never used the large model's signature skill — generating text. It did not need the model to write explanations or reason out new content; it only needed the model to pick one of 20 known options. So they turned to Strands Decider, a decision model. The difference between a decision model and a large language model can be put in one sentence: give it a state, a question, and a set of options, and it returns a probability for each option plus a confidence score. It cannot generate free-form text. In this setting, the state is the ticket and the options are the 20 labels. Task and model fit like a key in a lock.

The swap happened in two steps. The team started with the original Strands Decider 2B and ran it on a test set of tickets; then they found the E4B build based on Gemma 4 performed clearly better, so they switched to it. After switching, they tuned one more parameter: how confident the model had to be before a label was applied. The result, on a separate set of tickets from the ones used for testing, was that E4B beat production Sonnet. Note the wording: the blog says "beating," but it publishes no accuracy number. That is a gap worth remembering, and it comes up again below.

The numbers, line by line

The heart of the post is a comparison table, and it deserves to be read row by row. Start with time: average time per call was 4,215 milliseconds for Sonnet 4.6 and 897 milliseconds for Strands Decider E4B, 78% faster. Then tokens: per ticket, Sonnet 4.6 used 6,554 input tokens and 170 output tokens, 6,724 in total; E4B used 2,009 input tokens and 1 output token, 2,010 in total, 70% fewer. The output-token row — 170 down to 1 — is the most revealing line in the whole table. The old model had to generate a JSON answer with reasoning; the new one outputs a single choice and writes nothing extra.

Once tokens fell, cost followed. At published on-demand prices for the us-east-1 region: Sonnet 4.6 through Bedrock's global endpoint costs $3.00 per million input tokens and $15.00 per million output tokens; E4B runs on SageMaker ml.g5.xlarge instances — one A10G GPU with 24 GB of memory each — at $1.408 per instance-hour. On those prices, cost per ticket falls from $0.0222 to about $0.0005; per million tickets, from about $22,200 to about $500. The blog shows its working: at about 640 milliseconds per request, one instance handles about 5,600 tickets an hour, which would be $250 per million tickets at full utilization; keeping headroom for bursts, in practice it comes to about $500 per million. That is about 44 times cheaper than Sonnet — a 98% reduction.

One number is easy to miss: on the day Sonnet usage was switched off, average Lambda execution duration dropped 74%. The whole call chain got shorter — the model call was faster, so the Lambda ran for less time too. The blog is explicit that this cost math uses published list prices and leaves out Lambda charges (Lambda now runs for less time per ticket as well, so the real total is lower, not higher). But the other side of that coin matters too: Decider's cost depends on how busy each instance is. When instances sit idle, the cost allocated to each ticket rises; the "about $500" figure is the in-practice number with burst headroom, not the full-utilization ideal of $250. The blog prints both numbers, which is to its credit.

The blog gives two reasons tokens fell 70%. First, the output went from a JSON answer to a single choice, taking output tokens from 170 to 1. Second, the prompt got much shorter. Sonnet's production label descriptions were, on their own, longer than Decider's 4,096-token window — so the team rewrote each of the 20 labels as a one- or two-sentence criterion and sent them as the options of a single choice question, plus a "none" option. Long tickets get one more treatment: to fit the window, Decider drops the oldest comments and keeps the newer ones. That detail becomes a limitation worth flagging later.

Sonnet 4.6 versus Strands Decider E4B: cost per ticket $0.0222 versus about $0.0005, per million tickets about $22,200 versus about $500, average call time 4,215 ms versus 897 ms, total tokens per ticket 6,724 versus 2,010

The most copyable part is not the model — it is the three-way switch

If you only read the conclusion, this post is about "a small model replacing a big model." If you read the process, the most copyable thing in it is not the model at all, but a three-way switch. The team did not simply turn Sonnet 4.6 off. Instead, the labeling Lambda reads a three-way setting from AWS AppConfig on every message: BEDROCK, DUAL, or DECIDER. In BEDROCK mode, Sonnet labels every ticket and Decider is never called. In DUAL mode, both models are called in parallel: Sonnet's results are applied to the ticket and compared with Decider's. In DECIDER mode, Decider labels every ticket and Bedrock is never called. The team ran DUAL mode on live traffic for a while, checked every disagreement, and only then flipped to DECIDER. Rolling back is a 45-second config change.

That procedure is worth more than the 98% figure itself. Offline evaluation can only tell you how a model performs on a test set; DUAL mode runs on live traffic, real tickets, real distribution — including the edge cases no test set contains. The tickets where the two models disagree are exactly the ones worth a human look. Many teams replace a model by comparing it on a test set, deciding it looks close enough, and switching over directly. This team's approach was to let the new model shadow the old one on real traffic, judge it on real disagreements, and only then switch. That sequence is a methodology worth writing up on its own.

The infrastructure details are also worth recording, because they show what "swap in a small model" actually means in engineering terms. E4B runs on SageMaker ml.g5.xlarge instances, one A10G with 24 GB each; the model itself uses about 19 GiB. The endpoint autoscales with traffic, but its network is isolated, preventing outbound access from the container, and SageMaker copies the model weights in from S3 before start-up. Apart from Amazon's own ticketing APIs, everything in the path is a standard AWS service: ticket events land in an SQS queue that triggers a Lambda function, the Lambda calls a SageMaker AI Endpoint running Strands Decider, AppConfig holds the three-way switch, CloudWatch carries the logs, metrics, and alarms, and the whole stack is defined in CDK with deployment stages for Beta, Gamma, and Prod. In other words, this is not a bespoke system that only works inside Amazon — any team on AWS could build the same thing, minus the ticketing API.

There is one more pitfall in the post, stated with unusual candor: JSON key order changed the answer. The team found that sending the same content with keys in a different order moved output probabilities by up to 0.19, so they now send every request with sorted keys. 0.19 is not a small number — in a model that returns probabilities, it is easily enough to swap first and second place. That finding is a warning for any team serving a decision model behind an API: fix the request format before you compare runs, or you may be comparing serialization order rather than models.

How to read the 98% without being misled by it

98% grabs the eye, but the most important discipline in reading this post is not to let it carry you. First, draw the boundary around the number. It is calculated from list prices: Bedrock's published on-demand prices for Sonnet, SageMaker's instance price for Decider. Neither side includes Lambda charges, and neither reflects what Amazon's internal teams are actually charged — large-customer discounts and reserved-instance pricing are nowhere in the table. Second, it is an average cost per ticket that depends on instance utilization. The fuller the instances run, the lower the cost per ticket; in traffic troughs, idle instances push it up. The blog's two figures — $250 per million at full use, about $500 with headroom — show the number moves with utilization rather than standing still.

Third, and most important: the blog publishes no accuracy figure. It says E4B "beat" production Sonnet on a separate set of tickets and that the replacement "performed just as well," but it gives no accuracy, precision, or recall for either side. Without those numbers, a reader cannot independently judge what "just as well" means — 99% versus 98%, or 85% versus 84%? This is not nitpicking: classification quality is the premise the whole swap rests on. A 98% cost reduction only counts if quality did not drop, and the evidence that quality did not drop is currently the authors' own word. The authors built the system, so their word carries weight, but it is not third-party verification or a reproducible evaluation.

Fourth, "millions of dollars of token costs in a year" is a projection, not a measurement. The blog does not publish total ticket volume, so a reader cannot work backward from $0.0217 saved per ticket to an annual total. The arithmetic is transparent — savings per ticket times annual volume — but the annual volume multiplier is not public, so the projection cannot be checked. That does not make it wrong; it makes it a forecast based on internal data, and it should be read as one.

Fifth, two technical limits are buried in the details and deserve to be pulled out. One is the 4,096-token window: label descriptions had to be rewritten to one or two sentences to fit, and long tickets lose their oldest comments. That means Decider sees an incomplete ticket — if the decisive information happens to sit in the oldest comments, it never sees it. The other is the decision model itself: it can only choose, not explain. If the business one day requires an explanation attached to each label, or if the category list changes weekly, this approach stops applying. The authors say as much in their own lessons: this worked because the task is a closed choice from a known catalog, the volume is high, and the old path's latency genuinely mattered. If your labels need explanation text, if the catalog changes weekly, or if you cannot fit the useful context into a few thousand tokens, a decision model probably cannot replace the LLM's job.

This is not a "small model beats big model" story

It is easy to read this post as a crowd-pleasing tale of a small model defeating a big one, but that misreads what happened. What happened was not a contest but a correction in the division of labor. Sonnet 4.6 did not get weaker; it was doing a job that did not need it — a frontier model assigned to label tickets and generate a JSON reasoning trace for each one, like hiring a novelist to sort mail. Decider did not "defeat" anyone; it simply happens to be shaped like the job: take in a ticket, output a choice, write nothing extra. The 98% cost reduction does not come from the small model being smarter, but from the big model previously doing a lot of things this job never needed — a long prompt, 170 output tokens of reasoning per ticket — that were surplus to requirements all along.

That judgment has a second layer: the biggest savings did not come from the model getting smaller, but from the output getting shorter. Look at the table again: input tokens fell from 6,554 to 2,009, down 70%; output tokens fell from 170 to 1, down more than 99%. And output tokens cost five times as much as input tokens ($15 versus $3 per million). In other words, the largest share of the savings came from "no longer making the model write things it did not need to write." That is a prompt for every team using a large model for classification, labeling, or routing: first ask whether the task needs the model to generate text at all. If the answer is no, the first step to saving money is not a cheaper large model but a model that does not generate text — or at minimum, tightening the output format so the model stops writing a short essay for every choice.

The same week, Google published its Gemini Decisions API, built on the same idea: a model that only judges and does not write (VibeFix covered it on November 11). Two major vendors moving in the same direction in the same week suggests "decision models" are graduating from an internal optimization trick into a public product category. The logic is simple: the first wave of the large-model era was letting models do everything; the second is taking the tasks that never needed generation back from large models and giving them to smaller, faster, cheaper specialized ones. The Strands Decider story is one concrete sample of that second wave.

What to copy if you are doing classification with a large model

Start with the three things that transfer directly. First, copy the three-way switch. The BEDROCK, DUAL, DECIDER design is the most generally useful part of the whole post: the new model shadows the old one on live traffic first, disagreements are checked one by one, and only then does traffic switch over, with rollback taking 45 seconds. Any model replacement, large or small, is worth running in that order — it is closer to reality than any offline evaluation, and it can be reversed instantly if something goes wrong.

Second, copy the cost accounting. Break the bill into input tokens, output tokens, unit prices, instance-hour prices, and utilization, instead of quoting a single total. The reason this blog's cost table is credible is not that it arrives at 98%, but that it shows the source of every line: what the list prices are, what utilization is assumed, what full use versus headroom looks like, and what is left out. Run the same format on your own system; the answer may not be 98%, but at least it will be honest.

Third, copy the JSON key-order lesson. Before serving a decision model behind an API, fix the request format — sorted keys, field order, serialization — or the thing you think you are comparing may be formatting, not the model. That pitfall costs one line of code to avoid in advance, and days of debugging once you have fallen into it.

Now the three things to think through before copying anything. First, is your task a closed choice? A decision model only works when the options are known and stable: if the category list changes weekly, or labels need an attached explanation, it does not apply. Second, is your volume large enough? Decider's cost advantage comes from spreading an instance-hour price across a huge number of requests; at a few hundred tickets a day, idle instances may cost more than calling a large model's API. Third, can you live with not seeing the whole ticket? A 4,096-token window means long tickets get truncated, and the decisive information may sit in the part that was dropped; if your tickets are often long and the key information often comes early, evaluate that before deciding.

Finally, back to the 98%. It is a real number, and a bounded one: calculated at list prices, per ticket, in live operation with burst headroom, and premised on the authors' own statement that quality held. With all those qualifiers attached, 98% is still an enviable result — but it is a specific win on a specific task, not a slogan that "small models replace large models across the board." The real value of this post for readers is not the 98% but the method it demonstrates: see the task clearly first, then pick a model shaped like it; verify in shadow mode before switching traffic; do the arithmetic transparently before announcing the number. Follow that order, and whether or not you save 98%, at least you will not pay tuition on a model swap.

Views 8Comments 0

Comments (0)

ME
0/1000
Loading comments...

Sources

Browse projectsPublish your project

Related articles

A developer with glasses types on a laptop at a wooden desk, an external monitor to the left showing C code on a dark background, a small plant on the desk
News
Microsoft's Coding Model Stops Playing Second Fiddle: MAI Code 2 Scores 96% of Opus 5.5 for 5% of the Cost — Read the Three Caveats First

On 17 November 2026, Microsoft launched MAI Code 2 and Code 2 Flash: the larger model has 77 billion active parameters and a one-million-token context, averaging 63.2% across four benchmarks — close to GPT-6.1 Sol and Claude Opus 5.5 — at $0.309 per task. Read the three caveats first: promotional pricing, the lowest score of the three, and no long-horizon work.

Developer WorkflowProduct LaunchModel Updates
Against a black background, a conductor seen from behind raises a baton, with swirling trails of light circling beneath him like an orchestra being conducted
News
One API for 45 Models, and It Switches Providers When One Fails: Perplexity's Orchestrator Wants to Be the Switchboard

On 17 November 2026, Perplexity launched the Orchestrator API: one interface reaching OpenAI, Anthropic, Google, xAI and open-weight models such as GLM, Kimi and DeepSeek, billed at the answering model's published rate with no platform fee. When one provider fails, the same model retries on another deployment, and preview routing assigns requests to models by task.

Developer WorkflowProduct LaunchModel Updates
A developer types at a dual-monitor workstation; the left screen shows a dark code editor, the right a testing dashboard with green checkmarks and bar charts
News
Did Your Agent Get Better or Worse? AWS Open-Sources UAEF to Measure Agents from Every Framework with One Ruler

On 11 November 2026, AWS open-sourced UAEF: adapters turn traces from LangGraph, LangChain, Strands and Bedrock AgentCore into one format, then 36 metrics across 7 dimensions score them, flagging regressions past 5%. The judge model is hardened too — scores must cite verbatim evidence, and never decide safety.

Developer WorkflowProduct LaunchTesting & Quality