Liquid AI Releases LFM2.5-24B-A2B: Open-Weight 24B MoE Tuned for Fast On-Device Agents

On October 13, Liquid AI published LFM2.5-24B-A2B on the Hugging Face blog: a 24B-parameter mixture-of-experts (MoE) model with roughly 2B active parameters per token, released in both base and post-trained open-weight versions, positioned as "a reasoning model for on-device agents." Within days of release, the conversation around it converged on a single point: this model isn't competing on parameter count — it's competing on whether a small model can actually use tools well.

My take up front. Over the past year, the bottleneck for local agents was never "can it run" — llama.cpp, MLX, and Ollama pushed the bar down to a laptop long ago. The real bottleneck is this: hook a 20B-class small model up to an agent harness like OpenClaw or OpenCode, and it often can't even keep tool-call formats stable — multi-step tasks fall apart halfway through. What makes LFM2.5-24B-A2B genuinely interesting isn't the "24B parameters" figure, it's the training methodology: Liquid AI put the agent harnesses that vibe coders actually use every day — Pi, OpenCode, OpenClaw, Hermes Agent — directly into the final reinforcement learning stage. In other words, this model wasn't trained to game chat benchmarks; it was trained inside real agent workflows.

What it is, in one minute

LFM2.5-24B-A2B keeps the architecture of its predecessor, LFM2-24B-A2B: 40 layers, 64 experts per MoE block with top-4 routing, roughly 2B active parameters per token, and a hybrid of gated short convolutions and grouped-query attention (GQA). Three hard upgrades over the previous generation: the context window grows from 32K to 131K tokens, the vocabulary doubles from 65K to 128K via in-place tokenizer expansion (better tokenization for non-Latin scripts), and it is now a reasoning-only model — it produces an explicit chain of thought before every final answer.

Pre-training scaled from ~17T to ~33T tokens. Mid-training adds 2T tokens at 32K context (weighted toward reasoning, math, code, instruction following, and multilingual text) plus 300B tokens at 131K context (weighted toward long-context data). Tool calls and multi-turn trajectories run through both phases — laying the groundwork for the agentic training that follows.

The real story: five-stage post-training, with the last step done inside real agents

Post-training turns the base model into an agent in five stages:

1. Supervised fine-tuning (SFT); 2. Teacher specialization; 3. Multi-domain on-policy distillation (MOPD) on non-agentic tasks; 4. A second MOPD stage focused on agentic tasks; 5. Agentic RL — reinforcement learning inside real agent harnesses, with the environments named explicitly: OpenClaw, Hermes Agent, Pi, and OpenCode.

The first four steps are standard practice. The fifth is why this release deserves its own paragraph. Most models do RL on "evaluation tasks": give the model a question, check the answer, hand out a reward. Liquid AI instead dropped the model into agent frameworks and let it learn from calling tools, running multi-step tasks, and receiving feedback from real harnesses. Those two approaches train fundamentally different capabilities: the former is good at "getting the answer right," the latter at "getting the job done" — and that is exactly the lesson small models have been missing in local-agent scenarios.

Why? Because the capability decay curve is brutally steep for small models. A 70B-class model can flub a tool-call format once and reason its way back; a 20B-class small model that botches the first tool call watches its entire multi-step plan collapse. For small models, "can use tools" isn't a nice-to-have — it's the survival line. Liquid AI clearly gets this: rather than going head-to-head with large models on generic chat evals, they spent their training budget on making the small model "not make mistakes inside real workflows." That line of thinking is more worth copying than any parameter number.

Benchmarks: strong on agents and instruction following — but read the fine print

The official numbers come from 16 benchmarks run with vLLM, comparing LFM2.5-24B-A2B against Qwen3.6-35B-A3B (~50% more total and active parameters), Gemma-4-26B-A4B-it, and Nemotron-3.5-Lightning (30B-A3B). Where it wins:

On the 8 agent/tool-use benchmarks, it leads or ties on 5: AutomationBench at 27.69 vs Qwen3.6's 8.72 (more than 3x — a decisive lead), LiveClaw 48.09 vs 38.19, BFCLv4 65.01 (essentially tied with Qwen3.6's 65.00), ToolSandbox 80.71 vs 75.02, and τ³-Bench Banking 8.04 vs 7.63. It leads on all three instruction-following benchmarks: IFEval 96.75, IFBench 74.69, Multi-IF 84.38. That's a compelling combination: the two core competencies of an agent model — "can call tools" and "understands what you mean" — and it takes best-in-class on both.

But three things have to be said in the same breath, or this piece becomes a press release:

First, every benchmark is vendor-run. The blog states plainly: all models were evaluated with vLLM, using each benchmark's recommended generation parameters. These are Liquid AI's own numbers with no independent verification. The 3x blowout on AutomationBench is eye-catching — and precisely the kind of number that most needs independent replication. Until third-party evals land, treat these as vendor claims, not verdicts.

Second, its losses are concentrated — and the vendor admits it. Knowledge-heavy benchmarks: GPQA Diamond 74.42 vs Qwen3.6's 83.69, HLE 7.40 vs Gemma-4's 8.57. Heavy coding: SWE-Bench Verified (OpenCode) 51.00 vs Qwen3.6's 66.20, SciCode 32.71 vs 37.35. The blog's own recommendation, verbatim in spirit: pair it with retrieval or web search for knowledge-heavy workloads. That's an honest self-assessment — expecting a 2B-active MoE to memorize the world was never realistic. Its design philosophy is "reasoning and tool use happen locally, knowledge comes from outside."

Third, it didn't win PinchBench or WebArena Verified either. PinchBench: 83.18 vs Qwen3.6's 89.93. WebArena Verified: 26.75 vs 37.62. So the flip side of "leads or ties on 5 of 8" is that two of the remaining three are meaningful losses. Its agent capability has a profile: strong on automation orchestration (AutomationBench) and real-harness tasks (LiveClaw) — the "hands-on" scenarios — with a gap in web-browsing navigation (WebArena). Choose against your own use case, not the headline number.

How fast it runs: 184 tok/s on an M5 Max, 21K tok/s on a B300

Inference speed is the other hard number in this release, and it's specific. Liquid AI claims it's the fastest of the compared MoEs at the edge: 184 tokens/s decoding on an Apple M5 Max and 114 tokens/s on a Ryzen AI Max+ 395 — up to 1.7x faster than the next-best model while using the least memory. On GPU, it exceeds 21K output tokens/s at high concurrency on a single NVIDIA B300, the fastest in its size class.

Why does speed matter disproportionately for agent models? Because agents burn tokens at multiples of chat workloads: every turn produces a chain of thought first, then a tool call, then reads the tool output, then reasons again. A model with cheap per-token compute and fast decoding means the agent can stay "always on" without worrying about cost — the blog's own phrasing is "run agents continuously on your own hardware." That's the point of 2B active parameters: each reasoning token is cheap, so continuous operation stays affordable.

Ecosystem support is day-one complete: llama.cpp, MLX, vLLM, SGLang, and ONNX. Hooking it into a local agent is straightforward — serve the model behind any OpenAI-compatible backend, then point your harness at it. With Hermes Agent, for example:

hermes config set model.provider custom
hermes config set model.base_url http://localhost:8080/v1
hermes config set model.default LFM2.5-24B-A2B
hermes config set model.context_length 131072
hermes config set model.api_mode chat_completions
hermes config set agent.tool_use_enforcement true

The port depends on your serving backend: 8080 for llama.cpp and MLX, 8000 for vLLM, 30000 for SGLang, 1234 for LM Studio. OpenClaw and Pi configs are on the model card, and the full setup guide is in Liquid AI's official docs.

Who it's for — and who it's not for

Best fit: teams that want to deploy agents privately, where data can't leave the network. This is LFM2.5-24B-A2B's clearest target: finance, healthcare, government, or anyone sensitive about data exfiltration. These teams used to face an ugly choice — cloud frontier models (data leaves) or local small models (agent capability too weak to matter). An open-weight model with best-in-class tool use and instruction following at its size, decoding at 184 tok/s on a laptop, collapses that trade-off. 131K context is also enough room for a decent-sized agent working memory.

Second: local vibe-coding users. OpenCode and Pi users can point their local models at everyday coding-agent work. Mind the caveat: SWE-Bench Verified at 51.00 vs Qwen3.6's 66.20 means it's a clear tier down on heavy coding tasks. My suggestion: use it as the "daily chores agent" (config edits, scripts, toolchain calls) and hand serious coding work to a stronger code model — or wait for community fine-tunes.

Not for: pure knowledge Q&A or memory-heavy tasks. The GPQA and HLE scores say it plainly. The vendor's advice — pair with retrieval or web search — isn't politeness; it's the correct operating posture for this model: RAG plus LFM2.5 handling reasoning and tool orchestration, each doing what it's good at.

Closing: the second half for small models is "can do the work," not "knows more"

LFM2.5-24B-A2B represents a forming trend: the competitive center of gravity for small models is shifting from "parameters and knowledge" to "the ability to get things done inside real workflows." If third-party evals confirm Liquid AI's approach of doing RL inside real agent harnesses, it could become the standard training recipe for 20B-class agent models — the way SFT became standard once upon a time.

It's too early for conclusions, of course. All benchmarks are vendor-run and independent verification hasn't landed; the WebArena gap is a reminder that "agent capability" is a profile across scenarios, not a single score. But the direction is right: for anyone who wants to run agents locally and privately, there's now one more open option worth a serious evaluation — and the evaluation is cheap. The weights are already on Hugging Face, base and post-trained versions both, and llama.cpp could run them on day one. Run it yourself and see how it behaves in your own harness; that tells you more than any benchmark number.

One-line summary: don't stare at the 24B number — look at the training method. Putting the tools vibe coders use every day directly into reinforcement learning is the real story of this release.