Ian Paterson published a nine-phase experiment log this week covering what actually happened when he spent three months chasing decode speed on a pair of 3090 Tis. The punchline is worth knowing before you reach for DFlash on long-output workloads.
The Collapse
DFlash Attention speeds up decode — until output gets long. Paterson's numbers show DFlash dropping from 46.9 tok/s at 100 output tokens to 30.1 tok/s at 2000 output tokens. That's a 36% fall-off on the same model, same hardware, same prompt.
MTP (Multi-Token Prediction, the built-in speculative decoding variant in llama.cpp) with the right flags holds 39–49 tok/s flat across the same range.
The Flag That Changed Everything
The catch is that vanilla MTP looked dead at first. With n_max=3 and no probability filter, MTP delivered only 1.4× autoregressive — and was about 2× slower than DFlash on long workloads. Paterson buried it at Phase 2 and moved on.
What resurrected it was PR #22397 (merged April 28), which added --spec-draft-p-min to vanilla llama.cpp. The flag filters draft tokens by minimum acceptance probability, cutting the wasted cycles where the drafter produces low-confidence tokens that the verifier rejects anyway.
The winning config:
--spec-draft-max-n 6 --spec-draft-p-min 0.75
With those two flags, MTP went from buried to shipped. Final measured result: 1.8× autoregressive throughput, decode stable across every output length tested.
Numbers Side by Side
| Config | 100-token output | 2000-token output |
|---|---|---|
| DFlash Attention | 46.9 tok/s | 30.1 tok/s |
MTP (n_max=6, p_min=0.75) |
~39 tok/s | ~49 tok/s |
| Plain autoregressive | baseline | baseline |
At short output, DFlash still wins. The crossover is around 900 tokens. Below that, plain autoregressive is faster than MTP. Above that, MTP holds where DFlash doesn't.
What It Requires
MTP needs an MTP-variant GGUF — not the standard Q4_K_M you already have. The model file bundles the drafter weights. Paterson switched from a standard GGUF to an MTP-variant Q4_K_M and dropped the separate drafter file entirely. Migration was five minutes of wall time with zero client changes (same OpenAI-compatible endpoint shape).
One more flag worth noting from the same sweep: --reasoning-budget 256 saved about 10 seconds per request with no quality regression. It caps runaway reasoning chains. Paterson calls it the highest-impact, lowest-risk flag in the entire experiment.
The Catch
Speculative decoding is not bit-for-bit lossless at temperature=0 on free-form prose. Tool-call schema integrity is preserved, and the effect is documented academically (arxiv:2605.09992 on drafter attention drift), but if you're doing deterministic prose generation and need identical outputs on reruns, this is worth testing explicitly before shipping.
Also: MTP wins on wall clock above ~900 output tokens. For short completions, the overhead costs more than the speedup buys. Know your workload distribution before switching.
The Short Version
If you run llama.cpp for long-form output — summaries, drafts, extended reasoning — and you're currently on DFlash, your decode throughput is degrading as output grows. MTP with --spec-draft-p-min 0.75 and --spec-draft-max-n 6 doesn't. The flag has been in vanilla llama.cpp since April 28. The model file swap is the only real friction.
Sources: