Split the Work, 5x the Tokens: Cerebras Divides Inference and Gets Five Times the Throughput per Megawatt
On 17 November 2026 Cerebras said its disaggregated inference setup is now in production: AMD Instinct GPUs handle prefill, CS-3 systems handle decode, delivering 5x the throughput per megawatt of a CS-3-only setup at the same speed. The number is real but narrow — one shared endpoint, one vendor measurement.

In one sentence: split the work, get 5x the tokens from the same megawatt
On 17 November 2026, AI chip company Cerebras announced that its "disaggregated inference" setup is now in production: prompt processing goes to AMD Instinct GPUs, and answer generation stays on its own CS-3 systems. Measured on its own inference cloud, the combination delivers five times the throughput per megawatt of a CS-3-only configuration, with no loss in token-generation speed. The measurement came from a shared GPT-OSS-120B endpoint on 14 November, comparing the same endpoint before and after disaggregation was enabled. The interesting thing here is not the number "5x" itself, but the judgment behind it: inference is being split into two jobs, and which job runs on which kind of chip is becoming a business in its own right.
First, the context. Over the past year, agentic applications have kept getting faster and more demanding. Tools like Muse, Copilot and Codex need highly interactive responses — a user types a line and will not wait around. But data center power is finite; the megawatts a site can get are limited. So getting more users served out of the same power budget has become the most practical competition among inference providers. Cerebras's answer is: stop asking one kind of chip to do the whole job. Prompt processing and answer generation are two completely different kinds of work, and forcing the same hardware to do both wastes power.
Prefill and decode were always two different jobs
One inference request has two stages. The first is prefill: the model reads the user's whole prompt at once and computes intermediate state. This stage is highly parallel and compute-heavy — like settling an entire ledger in one go. The second is decode: the model takes that state and emits output one token at a time, reading model data from memory again with every step. This stage is serial, and its bottleneck is not compute but memory bandwidth — the faster it reads, the faster it emits.
That distinction decides the division of labor. GPUs are good at parallel computation, so prefill is their natural job; Cerebras's wafer-scale chips put tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth on a single chip, so during decode they do not have to keep fetching data from off-chip memory. According to Cerebras, a single chip carries 350x the SRAM capacity and 5,400x the memory bandwidth of a typical GB300 GPU. Because model layers and concurrent requests fit directly in on-chip memory, the CS-3, CS-4 and CS-5 can run decode at what the company calls ultrafast speeds. Moving prefill to GPUs lets the CS-3 devote all of its serving capacity to decode — that is where the 5x comes from. The chips did not get faster; the division of labor got more sensible.
The idea itself is not new. The industry has been discussing splitting prefill and decode for a while, and other splits — draft-target, attention-FFN — are also being tried. Cerebras explained disaggregated inference on 1 October 2026, and early tests then also reported a 5x gain. What is different about the 17 November post is that it offers a production measurement and a deployment package for partners, not just an experimental result. "We tried it and it is 5x faster" and "we already run it this way in production" are two different things.
How the 5x number was actually measured
First, what is credible about it. The measured object is a shared GPT-OSS-120B endpoint on the Cerebras Inference Cloud, and the comparison is the same endpoint with disaggregation switched off and on, with the baseline normalized to 1x. In other words, numerator and denominator are the same service and the same traffic, with only the disaggregation switch flipped — so 5x is a before-and-after comparison, not a case of pitting the company's best configuration against someone else's worst. Cerebras also states explicitly that token-generation speed did not drop. That matters: if speed had fallen, higher throughput would be useless, because agentic applications need speed, not just volume.
Now the limits, which Cerebras itself states in the blog — they should not be skipped over. First, the baseline is a "CS-3-only configuration", not a GPU setup and not another vendor's inference cloud. 5x means "split is 5x faster than unsplit", not "Cerebras is 5x faster than everyone else" — those are very different sentences. Second, the measurement comes from a single shared endpoint and one sampling on 14 November, with no absolute tokens-per-second or power figures. Without absolute numbers, outsiders cannot independently recompute the result; they can only trust the normalized ratio. Third, this is the vendor's own measurement, with no independent verification. Unite.AI noted in same-day coverage that the central question is whether the advantage measured on one endpoint can be reproduced across partners' chosen accelerators, frameworks and deployment sizes.
So the right way to read 5x is: it proves the disaggregation route works in Cerebras's own production environment, but it is not yet an industry scorecard. What would make the number stand up is deployment data from partners over the next year. If AWS, Fireworks AI and the other early partners report similar multiples, 5x is real capability; if only Cerebras's own cloud can produce it, it looks more like a polished product demo.
Not just its own cloud: two ways for partners to plug in
This release is not only about Cerebras's own cloud; it also ships a deployment package for infrastructure partners. Partners can follow a Cerebras-provided reference architecture and co-locate their chosen prefill accelerators and Cerebras systems on a unified network. There are two software integration routes: an end-to-end disaggregated software stack for partners optimizing time-to-market, or a decode-only SDK for partners optimizing for control.
The end-to-end stack has four parts: a disaggregated inference framework that routes requests between prefill and decode pools and manages replica capacity and health; a prefill engine that processes prompts using open-source inference engines like vLLM or SGLang; a Cerebras decode engine for response generation on Cerebras systems; and software called KV Portal, which transfers intermediate model state — including key-value caches — between the two engines over RDMA. The decode-only SDK is the opposite: partners keep their own inference framework and prefill engine and connect only to the Cerebras decode engine and KV Portal, plus development aids such as a mock Cerebras decode replica and a reference integration with an open-source disaggregated inference framework and prefill engine.
The "two routes" design itself says something about Cerebras's position. It wants to sell not just inference service on its own cloud, but "decode acceleration" as a capability. The early-access partners named are Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute, and Cerebras says it is expanding access to more partners. The list includes a cloud giant and inference providers, which means Cerebras does not plan to fight Nvidia GPUs head-on for compute; it wants to be a "decode plug-in" inside other people's data centers. If that positioning works, its competitors are no longer just chip companies, but every software stack that optimizes inference.
700 megawatts and hundreds of megawatts: ambition and open questions
Beyond 5x, two scale numbers in the blog are worth noting. Cerebras says it plans to deploy disaggregated inference broadly across hundreds of megawatts in its own inference cloud and partner deployments by the end of 2027. It also mentions the recently announced CS-4, dramatically scaled manufacturing, and a planned 700MW of data center capacity. Hundreds of megawatts is a deployment target, not capacity already running; 700MW is planned capacity, not built-out floor space. Their significance is that they turn "5x" from one experiment into a product line: Cerebras is betting that inference demand over the next two years will be large enough to need hundreds of megawatts of new capacity, and that a good share of it will run disaggregated.
Whether that bet is reasonable depends on two things. First, whether agentic applications really keep growing. Cerebras cites Goldman Sachs research to show agent demand is rising, but rising demand does not mean it all runs on Cerebras. The inference volume behind Muse, Copilot and Codex is indeed growing, but the compute buying behind them is spread out — who wins orders depends on price, speed, reliability and ecosystem, not on one 5x figure. Second, whether the CS-4 and CS-5 can ramp on schedule. The blog calls the CS-4 "recently announced", but gives no delivery timetable or customer list. Ramping a wafer-scale chip has never been a paper exercise: yield, delivery and customer validation can each slip, and if the new chips slip, the hundreds-of-megawatts target will have to be carried by the CS-3 alone.
There is also a more practical issue: disaggregated inference makes the system more complex. With prefill and decode running separately, key-value caches have to move between them, adding a layer of networking, scheduling and fault isolation. KV Portal moves state over RDMA, which sounds fast, but latency and stability across vendors and sites are unproven. The reference architecture and the mock decode replica are there to lower that barrier — but lowering a barrier is not the same as removing it. How much integration really costs will only become clear from the first partner deployments.
The wafer-scale bet: what this hardware route is gambling on
To understand why Cerebras dares to sell decode as a standalone business, you have to look at its hardware route, which is completely different from everyone else's. Mainstream GPUs assemble many small chips; Cerebras goes the other way and builds one chip out of an entire wafer — the wafer-scale engine. One chip is one whole wafer, covered in compute cores and SRAM, with cores talking to each other without ever leaving the die. The benefit is obvious: memory bandwidth so large that data does not have to shuttle between chip and memory. The drawback is equally obvious: building a chip that big makes manufacturing, yield and cost all hard problems, since any defect on the wafer can drag down the whole chip.
That is exactly why its decode advantage holds. Decode is a memory-bandwidth game, and wafer-scale chips are built for memory bandwidth. The 350x and 5,400x figures in the blog compare against a "typical GB300 GPU" on single-chip on-die SRAM capacity and memory bandwidth. That comparison needs care: it compares Cerebras's strongest dimension, not overall performance. GPUs keep their advantages in general-purpose compute, ecosystem and programming models, which wafer-scale chips do not have. So the two multiples mean "on decode as a single discipline, Cerebras's hardware form wins" — not "Cerebras chips beat GPUs across the board". In fact, Cerebras itself concedes the GPU's value this time: prefill goes to GPUs, which is a public admission that outside its own strength, the GPU is still the better choice.
This "each does what it is best at" stance reflects a broader shift across the AI hardware industry. People used to compare single-chip benchmark scores; now more and more vendors admit that no one chip can do everything. Training, prefill, decode and vector retrieval each have different bottlenecks, and forcing one kind of hardware through the whole pipeline wastes something somewhere. Cerebras's disaggregation is this thinking made into a product: it no longer tries to prove wafer-scale chips can do everything, but positions itself as a "decode specialist", leaving prefill to GPUs and system integration to partners. It is a more pragmatic play — but it also ties its fate to partner adoption. If early partners like AWS and Fireworks AI find integration too painful, or the multiples do not match expectations, this route will move slowly.
One more piece of context: Cerebras did not publish this blog in a vacuum. In the same month, Nvidia reported quarterly revenue of $115.8 billion, with data centers at 93% of sales — the AI compute market is still Nvidia's world. Cerebras's 5x is a voice raised inside a giant-dominated market. It does not need to convince everyone, only the customers who are already worried about inference cost and interactive speed: for the same power budget, we can serve 5x the requests. Whether that pitch holds will be decided by customers' ledgers, not by the blog's wording.
What to watch over the next year
After this blog post, the numbers worth tracking are not 5x itself, but the ones that can verify it. First, partner deployment feedback. Will the four early partners — AWS, Fireworks AI, Gimlet Labs and General Compute — publish their own disaggregated inference data in the coming quarters? If they stay silent, integration is probably harder than expected or the results are weaker; if they report multiples close to 5x, the route has independent backing. Second, CS-4 production ramp. The blog calls the CS-4 "recently announced" but gives no delivery schedule or customer names. Ramping wafer-scale chips is never trivial, and the hundreds-of-megawatts target depends on the CS-3, CS-4 and CS-5 arriving in sequence. If CS-4 slips into late 2027, that target gets very tight.
Third, absolute performance data. Cerebras has still not published absolute tokens-per-second or power numbers for the disaggregated setup, so the industry cannot put it on the same table as other inference services. The day it is willing to publish absolute figures is the day it is ready for head-to-head comparison. Finally, one question matters for developers: will disaggregated inference become the industry's standard practice? If yes, the benefit spreads beyond Cerebras — every inference optimizer will follow, and open-source engines like vLLM and SGLang will make splitting smoother. Then Cerebras's first-mover edge dilutes, and it will have to rely on the decode speed of the CS line itself and the scale of hundreds of megawatts of deployments. If no, and splitting only pays off on Cerebras's own cloud, it remains a niche but profitable market. Either outcome is good for developers: inference gets cheaper, or there is one more architecture to choose from.
What this means for developers
For an ordinary developer, the direct meaning of this news is limited: the inference service you use today will not get cheaper because of this blog post, and Cerebras published no prices. Its meaning sits on a longer timeline. If disaggregated inference really runs at hundreds of megawatts, the unit cost of inference could come down, and agentic applications could afford to be "always on" more cheaply. Conversely, if the 5x only holds on Cerebras's own cloud, its impact on the industry is just a reminder about a technical direction: splitting prefill and decode is worth taking seriously.
More worth noting is the industry signal this news carries. The inference race is shifting from "whose chip is faster" to "whose system divides labor better". Models are getting bigger and agents more capable, and piling on compute alone is no longer enough. Splitting prompt processing, decode, caching and scheduling and handing each to the hardware best suited for it is becoming the core capability of inference vendors. Cerebras's bet is "GPUs do prefill, wafer-scale chips do decode" — a clear division of labor, but whether it is the best one will depend on what rival divisions of labor other vendors bring out over the next year.
Overall, the credibility of this blog post is above average: the measurement method is stated clearly, the scope and limits are not hidden, and the partner list and deployment targets are on the table. Its weak points are equally clear: one endpoint's self-measured data, no absolute numbers, no third-party verification. 5x is a number worth remembering — but remember its qualifiers too: on the Cerebras Inference Cloud, one shared GPT-OSS-120B endpoint, before-and-after disaggregation, 5x throughput per megawatt with no loss in speed. Only when partner deployment data arrives will that number truly land.
Comments (0)
Sources
Related articles
On 20 November 2026, OpenAI released MedChemBench: 99 medicinal-chemistry problems from 117 papers and patents, covering 761 molecules. GPT-6.2 Astra scores 84.5% overall — but 95.4% on assembling structures and only 50.4% on choosing compounds. The total flatters; the components tell the truth.
On 18 November 2026, Google Research and co-authors posted BuggyMLE to arXiv: bugs injected into 222 working ML repositories, and 14 models slide from 95% on the easiest tier to 9% on the hardest. A 9-billion-parameter model trained easy-to-hard beats Claude Opus 5.5 on the 10-bug tier.
On 19 November 2026, OpenAI disclosed its internal Performance Factory: Codex agents automatically investigate ChatGPT performance regressions, running 1,700+ investigations in three months and finding 600+ root causes. Yet only about 15 fixes were merged exactly as Codex wrote them — agents can take over the investigating, but fixing still needs a human.