MTP in llama.cpp Adds 24–33% Throughput on Qwen3
llama.cpp merged Multi-Token Prediction for Qwen3. Community benchmarks show 38→47 tok/s on RTX 3090 and 63→84 tok/s on RTX 5090 — no new hardware needed.
#llama-cpp
3 posts
DFlash Decode Collapses at Long Output. MTP Doesn't.
One llama.cpp flag—--spec-draft-p-min 0.75—turns MTP from a dud into a decode speed that holds flat across output lengths where DFlash falls apart.
Speculative Decoding Explained: Why Your Local Model Got 2× Faster in 2026
The same Qwen3.6-27B that ran at 70 tokens/sec on a 4090 in January was running at 140 tokens/sec by April. Nothing changed about the model. Speculative decoding moved from research curiosity to default. Here is what it actually does.