llama.cpp merged the Multi-Token Prediction (MTP) branch last week. If you run Qwen3 locally, this is worth testing now.

MTP lets the model draft and verify multiple tokens per forward pass instead of one. The arithmetic stays the same; you just get more tokens out per wall-clock second when the draft heads agree with the main model.

What the numbers look like

Community benchmarks on Qwen3.6-27B Q4_K_M, reported across several GPUs:

GPU Without MTP With MTP Delta
RTX 3090 38 tok/s 47 tok/s +24%
RTX 5090 63 tok/s 84 tok/s +33%
M5 Max (Apple Silicon) ~31 tok/s ~37 tok/s ~+19%
RTX 4090 (HN report) ~120 tok/s avg ~150 tok/s peak +25%

These are from independent testers running the same model and quant with identical prompts, not a vendor benchmark page. The 4090 numbers are from a Hacker News thread from the same day the merge landed; take the precise figures there with a small grain of salt, but the direction is consistent across hardware.

How to enable it

Pull the latest llama.cpp from main (the MTP branch was merged, so any build after mid-May 2026 should include it). No new flags are required for Qwen3 — the model's MTP heads are detected automatically from the GGUF metadata.

Verify you're getting MTP by watching the decode speed on the first response versus a warmed-up query. If MTP is active, you'll see a noticeable jump after the first few tokens.

For llama-server:

./llama-server \
  -m qwen3-27b-q4_k_m.gguf \
  --ctx-size 8192 \
  --n-gpu-layers 99

No extra flag needed. MTP activates automatically when the model file contains the draft heads.

What it doesn't fix

MTP throughput gains are speculative-decoding style: they depend on how well the small draft heads predict the main model's output. On highly constrained or repetitive tasks the gains hold up; on creative or low-temperature outputs where the model surprises itself, the acceptance rate drops and gains shrink.

Context length also affects it. The ~37 tok/s M5 Max number is at shorter contexts; at long contexts it will drift down, same as baseline.

Memory overhead from MTP heads is small — the draft heads add a few hundred MB to VRAM, not a full copy of the model.

Worth testing if you use Qwen3 for agents

For agentic loops where the model is generating structured output (tool calls, JSON, short reasoning steps), the acceptance rate on draft heads tends to be high. That's exactly where local throughput matters most — long chains of fast, cheap inference steps. A 24–33% speed increase there translates directly into faster agent cycles with no cost change.

I haven't run this on my own hardware yet — these numbers are from community testers — but the hardware range is broad enough that the pattern is credible. Pull latest llama.cpp and run your own baseline before and after.

Sources: