Baidu's ERNIE 5.5 Flash Base Is a 280B Model You Can Download — If the License Lets You

First, the verdict: this is a "half-finished release" — and the half-finished part is the point

On November 5, 2026, Baidu released ERNIE-5.5-Flash-Base, a sparse mixture-of-experts model with 280 billion total parameters and only 14 billion activated per token. It was pre-trained from scratch on 33.5 trillion tokens, supports a 1-million-token context, and its weights are downloadable from Hugging Face. But the strangest thing about this announcement is what the model cannot do: it has had no instruction tuning, no reinforcement learning, no chat alignment, and no multimodal ability at all. It does exactly one thing — continue text.

My verdict up front: the real product here is not the model itself, but the offer to skip pre-training. Baidu has done the most expensive, most time-consuming stage and left behind a clean checkpoint, so others can build on it — domain fine-tuning, instruction training, reinforcement learning, agent training. The idea is not new, but the timing lands on three live trends: cheap models are becoming the default, open-weight licenses are getting tighter, and pre-training from scratch no longer pays off for most teams. Once you see those three, the benchmarks and the license in this announcement start to make sense.

What the announcement actually says

Facts first, no interpretation. According to Baidu's official release page, ERNIE-5.5-Flash-Base is a sparse mixture-of-experts language model with 280 billion total parameters and 14 billion activated per token. It was pre-trained from scratch on 33.5 trillion tokens with the Muon optimizer and FP8 mixed precision, with training context gradually extended to 128K tokens, and fine-tuning and inference support a 1-million-token context. The released weights are the checkpoint at the end of pre-training, before any mid- or post-training, meant as a common starting point for domain adaptation, instruction tuning, and reinforcement-learning research.

Architecturally it has 43 Transformer layers, a hidden size of 4096, and 64 attention heads. Every layer except the first is a mixture-of-experts block: 512 routed experts with top-10 activation, plus one shared expert. For attention, most layers use Heavily Compressed Attention (HCA), compressing every 128 tokens into a single cached entry while keeping a short window of recent uncompressed tokens; only one layer in seven keeps token-level access to the full context, using multi-head latent attention (MLA) plus sparse attention. The output head uses MultiMax, a learnable piecewise temperature that replaces a single softmax temperature, and the multi-token prediction module is replaced by ECHO, which reuses the backbone's own final layer to predict future tokens. The tokenizer is a new BPE with a vocabulary of about 200K entries, tokenizing numbers right-to-left in groups of up to three digits, which Baidu says helps with arithmetic and number-heavy text.

On benchmarks, Baidu evaluated ERNIE-5.5-Flash-Base against three base models of similar size using the same evaluation harness, prompts, few-shot examples, and the same LLM judge for free-form questions: DeepSeek-V4-Flash-Base (284B total, 13B active), Tencent's Hy3-preview-Base (295B, 21B active), and Xiaomi's MiMo-V2.5-Base (310B, 15B active). Across 30 benchmarks, Baidu reports ERNIE finishing first in 24. The widest gaps are in code: HumanEval+ 82.3 versus DeepSeek's 69.5, MBPP+ 81.5 versus 67.2, LiveCodeBench 40.1 versus 31.9; on exams, MMLU 93.3 versus 89.4 and MMLU-Pro 78.4 versus 69.2; on long context, RepoQA 91.7 versus 81.0. It also loses some: ChineseSimpleQA 73.0 against DeepSeek's 75.3, BBH with chain-of-thought 89.2 against Hy3's 90.0, FullStackBench 54.6 against Hy3's 56.7, and InfiniteBench 46.5 against DeepSeek's 48.0. One caveat applies to every number in this paragraph: all of them are Baidu's own reported results, with no independent third-party verification yet, so treat them as vendor-reported throughout.

For access, the weights are on Hugging Face in FP8 and BF16 safetensors formats, and both SGLang and vLLM can serve the model through the completions API. Baidu also says a post-trained ERNIE 5.5 Flash will follow — this Base release is only the starting point.

The architecture: every design choice is in service of "cheap"

The technical section of this announcement is worth reading closely, because it lays out the cost arithmetic of a cheap large model. Baidu states the design goal up front: a base model meant for further training has to stay affordable to train and run. Further training samples from the model repeatedly, and agentic applications bring multi-turn interactions and long contexts, both of which push up compute and memory costs. So ERNIE 5.5 Flash revisits the trade-off between capability and efficiency in every component — attention, residual connections, MoE layers, output head — with no part optimized for performance alone.

The attention design makes the point clearest. Of the 43 layers, 37 compress every 128 tokens into one cached entry, shrinking KV-cache demand by roughly an order of magnitude, at the cost of those layers seeing only a coarse view of distant history. The remaining six layers keep full context but use sparse attention, computing core attention only over the top-k highest-scoring positions for each query, so per-query cost stays constant as context grows. Baidu reports that this HCA-plus-MLA combination beats HCA-plus-compressed-sparse-attention at a similar KV-cache budget in its pre-training comparisons. In other words, long-context ability here is not bought with more VRAM — it is bought with a deliberate choice about which layers deserve to see the full text and which can settle for a summary.

The MoE layers make a similar trade. A conventional setup uses one router and a hard top-k, but Baidu found that a single router's scores correlate only weakly with how useful each expert actually is, and experts left out of the top-k receive no gradient, so wrong selections reinforce themselves. Switching to several independently initialized sigmoid routers that score experts jointly reportedly improves the correlation between routing scores and expert usefulness noticeably, with the largest gains in the middle and deeper layers — while the number of activated experts, expert FLOPs, and communication volume stay the same. The experts themselves run in a 2048-dimensional latent space, which Baidu reports roughly halves dispatch and combine traffic and expert FLOPs relative to full-width experts.

One more detail worth noting: the training system was designed together with the model. The Muon optimizer orthogonalizes each parameter matrix's update as a whole, which changes distributed communication completely versus Adam, so Baidu built communication-efficient update paths and memory-balanced sharding for it. The FlashMask-4 attention kernel targets large head dimensions with 2-CTA tiled parallel computation, and Baidu reports it running up to 175% faster forward and 74% faster backward than FlashAttention-4's MaskMod on SM100 GPUs. Those figures are vendor-reported too, but they point to a more important judgment: this model's cost advantage does not come from a single trick. It comes from architecture, data, and training infrastructure all squeezing cost at the same time. Any single number overstates or understates it; taken together they tell a complete cost story.

How to read the benchmarks: what 24 out of 30 is actually worth

Baidu's self-reported "first in 24 of 30 benchmarks" sounds dominant, but reading benchmarks means looking at where it loses, where it wins, and who it beat. Start with the opponents: DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base are all base models of similar size from the second half of 2026, evaluated with the same harness, the same prompts, and the same LLM judge. That shared setup itself deserves credit — cross-comparisons of base models are most often ruined by everyone running their own pipeline, and Baidu at least made the scores comparable.

Then look at where it wins. The biggest gaps are in code and exams: HumanEval+ 82.3 versus 69.5, MBPP+ 81.5 versus 67.2, MMLU-Pro 78.4 versus 69.2. These three share a common cause, and it is probably the data rather than the architecture: the announcement stresses that the corpus goes beyond static code to include commits, issues, pull requests, code reviews, requirement-to-PR trajectories, generated tests, and CI-verified fixes, explicitly targeting repository-level reasoning and agentic software engineering. A base model leading code benchmarks by 13 points is most likely a data-mix achievement, not an architecture achievement.

Then look at where it loses. Of the six losses, three are in long-context and multilingual precision work: HELMET-Rerank 56.2 against DeepSeek's 60.0, InfiniteBench 46.5 against 48.0, INCLUDE-lite 79.5 against 79.8. The gaps are small, but they cluster in exactly one place — fine-grained operation over very long contexts. That lines up with the architecture trade-off: 37 layers only see compressed history, so precise long-range work is precisely what was traded away. Losing BBH with chain-of-thought and FullStackBench to Hy3 also shows this is not a model that leads everywhere, but one that leads in the directions Baidu chose.

So the right reading is: 24 out of 30 says this base has a solid foundation, especially on code and exam-style tasks — but it is not a chat model you can use out of the box. Between a base model's benchmark scores and a finished product's experience sits an entire post-training stage, and Baidu says so plainly in the announcement itself: this checkpoint is only the starting point, and the post-trained version comes later.

The license is the real news in this announcement

If the benchmarks are the headline selling point, the license is the part of this release genuinely worth arguing about. ERNIE-5.5-Flash-Base is not released under Apache 2.0. It ships under Baidu's own custom agreement, version 1.0, dated November 5, 2026. The previous generation, ERNIE 4.5, used Apache 2.0. This generation switches to a custom license, and the conditions are tight.

First, territory. The agreement's "Territory" explicitly excludes the European Union, the United Kingdom, and South Korea — use, modification, and distribution there are not authorized. Second, a revenue threshold. Commercial use is free of model license charges while the annual revenue of related products or services stays at or below US$50 million over the preceding 12 consecutive months, provided you send a one-time use notice and display "Powered by ERNIE" prominently in the product. Above US$50 million, you need Baidu's separate written commercial authorization before starting or continuing that use, and commercial use without it must be suspended once the threshold is crossed. Third, a restriction on improving other models. The agreement forbids using the model, its derivative models, or their outputs to improve any other AI model or machine-learning system. Fourth, verification. Baidu may request a written compliance statement and supporting information, no more than once per calendar year per operator, from operators at or above the US$50 million threshold.

This license only makes sense inside the open-weight landscape of late 2026. In the same week, Tencent shipped the full Hy4 but held the weights back, promising them "in the coming days"; Moonshot opened the weights of Kimi K3.1 Pro; Cognition's SWE-2 Flash does not publish weights at all. Labs are splitting in different directions: some ship the API first and the weights later, some ship weights under tightening licenses, some do not ship weights at all. Baidu's choice is a variant of the middle path — the weights are out, but a custom agreement fences in who can use them, where, and up to what revenue.

For vibe developers, the practical consequences are concrete. If you are building a personal project, doing research, or running a small product with under US$50 million in related revenue in mainland China or the United States, the terms are not harsh: send one notice email, display "Powered by ERNIE," and commercial use is free. But if you are in the EU, the UK, or South Korea, or your product already earns more than US$50 million, these weights are effectively unavailable to you. The "do not use it to improve other models" clause is also worth a pause: it rules out distilling ERNIE's outputs into your own smaller model, which cuts off a workflow many teams rely on — using a big model to teach a small one. Read the license before downloading. This time that is not boilerplate advice; it genuinely decides whether you can use the model at all.

Seen against November's release wave: base models are becoming a "half-finished" business

ERNIE 5.5 Flash Base is not November's only half-finished release. In the same week, Cognition shipped SWE-2 Flash — cheap, but with weights withheld; Tencent shipped the full Hy4, with weights still to come; Baidu has now shipped a pre-training checkpoint, with the post-trained version still on the way. Put together, the trend is clear: frontier labs are slicing model releases into smaller steps — base first, post-trained version later; API first, weights later; or the reverse, weights first, post-training later. One release no longer equals one finished product. It equals one node on a timeline.

That split cuts both ways for vibe developers. The good side: public base models mean small teams no longer have to pre-train from scratch. Pre-training on 33.5 trillion tokens is a cost almost no small company can carry, and Baidu has done it and given it away — the most expensive step becomes a public good. Domain fine-tuning, agent training, and training-method research can all start from this checkpoint instead of burning months of budget first. The bad side: a half-finished model needs you to finish the post-training yourself, and post-training quality decides the final experience. A base model leading benchmarks by 13 points does not mean your fine-tuned product leads by 13 points — the gap has to be closed with your own data and training methods.

One more signal worth watching: the license is becoming part of the model release itself. Reading a model launch used to mean checking parameter counts and benchmarks. Now it also means checking the license's territory, revenue threshold, and improvement restrictions. The move from Apache 2.0 for ERNIE 4.5 to a custom agreement for 5.5 Flash says something on its own: open weights are moving from "unconditional openness" toward "conditional openness," with the conditions written into legal text rather than blog slogans. When choosing a base model, the license matters as much as the benchmarks, if not more — the benchmarks decide how good it is, the license decides whether you are allowed to use it.

Four suggestions for vibe developers

First, read the license before downloading the weights. Confirm your use is outside the excluded EU, UK, and South Korea territories, confirm related product revenue is under US$50 million, and confirm you do not plan to use its outputs to train other models. If any of those fails, get written authorization from Baidu first — do not download first and sort out paperwork later. Before commercial use, remember the one-time use notice and the "Powered by ERNIE" attribution.

Second, do not use it as a chat assistant out of the box. It is a base model with no post-training whatsoever: it will not follow instructions reliably, will not refuse inappropriate requests, and has no tool-use format. If you want to build a product on it, do instruction tuning or reinforcement learning first, or wait for Baidu's post-trained version. Shipping a raw base model to users is unfair to them and to yourself.

Third, if you fine-tune, start with code and agent directions. Baidu's self-reported benchmarks show the largest leads in code, and the corpus investment in repository-scale code trajectories is the most explicit. Building a code assistant, a repository-level agent, or automated test generation from this checkpoint follows its strengths. Chinese knowledge QA and fine-grained ultra-long-context work are its relative weaknesses — do not start there.

Fourth, do not treat 24 out of 30 as a selection conclusion. The benchmarks are Baidu's own runs, the three comparison models are all base models too, and independent verification has not landed yet. Before choosing a base, run a small comparison on your own tasks: the same fine-tuning recipe starting from ERNIE and from the base you use today, comparing final quality, training cost, and inference cost. Your own data beats any benchmark, and the license constraints belong in the total cost too.

One-sentence summary

ERNIE-5.5-Flash-Base is valuable not because it is a strong model, but because it finishes the pre-training step, gives it away, and attaches a clearly written bill: the weights are downloadable, but territory, revenue, and how you may improve other models all come with conditions. Its self-reported benchmarks — first in 24 of 30, with code and exams as strengths and fine-grained long-context work as a weakness — make it a starting point worth evaluating seriously for individual developers and small teams, while users in the EU, UK, and South Korea and products earning over US$50 million are effectively out of scope. Read the license before downloading, decide what post-training you need to add before fine-tuning, and only then judge whether this base belongs in your workflow.