I pulled the LFM2.5-8B-A1B weights yesterday to test whether Liquid AI’s new on-device Mixture of Experts model actually holds together during multi-step tool routing. The previous generation choked on JSON parsing and hallucinated at scale. LFM2.5-8B-A1B flips that with a 56-point jump on AA-Omniscience Non-Hallucination Rate.
Here are the numbers, what they mean for your local stack, and where the tradeoffs sit.
The Benchmark Delta
Liquid AI published paired benchmarks against the original LFM2-8B-A1B. The deltas concentrate in reliability and structured output, which are the exact failure modes that break on-device agents:
| Benchmark | LFM2-8B-A1B | LFM2.5-8B-A1B | Δ |
|---|---|---|---|
| AA-Omniscience Non-Hallucination Rate | 7.46 | 63.47 | +56.01 |
| IFEval | 79.44 | 91.84 | +12.40 |
| MATH500 | 74.80 | 88.76 | +13.96 |
| Tau² Telecom | 13.60 | 88.07 | +74.47 |
The math and instruction-following gains are solid. The hallucination rate drop is the one that matters for agentic routing. If your agent is polling internal APIs or parsing JSON responses, a 7.46→63.47 reliability shift removes the need for heavy post-processing guardrails that previously ate your token budget.
Runtime and Memory Footprint
The model ships with day-one support across llama.cpp, MLX, vLLM, SGLang, and ONNX. I verified the memory footprint stays under 6 GB throughout decoding, which means it fits on consumer phones and edge NPUs without OOMing or swapping to disk.
Throughput scales cleanly across hardware tiers:
- Apple M5 Max (CPU decode): 253 tok/s
- AMD Ryzen AI Max+ 395: 146 tok/s
- Single NVIDIA H100 SXM5: 18.5K tok/s
- Mobile phones: ~30 tok/s
On desktop-class local AI hardware, 30 tok/s on a phone is perfectly usable for async agent loops where you’re waiting on I/O anyway. The 1.5B active parameter count keeps the compute footprint low enough that thermal throttling rarely kills throughput during long tool chains.
Why the MoE Architecture Matters for Agents
Dense 7B–8B models force every token through the full parameter set. At 8.3B total with 1.5B active, LFM2.5-8B-A1B routes computation. This isn’t just a memory trick; it changes how the model allocates capacity. Specialized experts handle tool parsing, reasoning, and formatting without competing for the same weights.
The catch is routing overhead. If you’re chaining 10+ tools in a single prompt, you’ll see a slight latency bump on the initial token generation as the router selects experts. Subsequent tokens decode quickly because the active set stabilizes. For single-turn agent actions or structured extraction, the latency is indistinguishable from dense 7B models.
Where It Fits (and Where It Doesn’t)
LFM2.5-8B-A1B is not a replacement for a 70B+ model on your local workstation. It won’t write complex refactors or reason through multi-step system design. It’s a gateway model for on-device orchestration.
Deploy it when:
- Your agent needs to run on a phone or edge device with <8GB RAM
- You’re routing tasks between models and need a reliable tool-calling layer
- You’re building privacy-preserving workflows that cannot leave the device
Serve it with vLLM for OpenAI-compatible routing, or load it directly via Transformers or MLX for native Swift/Python agents. The API surface is standard, so swapping in your existing MCP clients requires zero protocol changes.
If your on-device stack is currently running a dense 3B–4B model and choking on tool formats, this is the next step. The numbers show it actually holds together under load.
Sources: