The standard assumption in deployment is that a 4-bit quantized model is a slightly worse version of its full-precision original. Multiverse Computing's new paper, Quantization-Aware Healing (QAH), challenges that directly: applied to a GPT-OSS 120B model compressed to 60B and quantized to MXFP4, the 4-bit result beats its own bfloat16 source on 7 of 9 benchmarks. The smaller model ends up both cheaper to run and more accurate than the checkpoint it was quantized from.
Why Healing a Compressed Model Is Different
The typical efficiency pipeline is compress-then-heal: structurally cut parameters (removing layers, heads, or neurons), quantize the weights to 4 bits, then run a recovery step. That recovery step usually means quantization-aware training (QAT), which inserts fake-quantization operators into the forward pass and keeps fine-tuning on a task loss. The problem: after structural compression, there is no independently trained full-precision version of the smaller architecture to learn from. The only candidate teacher is the recovered bfloat16 checkpoint — itself a distilled approximation of the original. Distilling from it caps the quantized student at that recovered checkpoint's own ceiling.
QAH removes that ceiling with one change: it distills directly from the original, pre-compression model as a frozen teacher. Teacher and student don't even share an architecture. The teacher is full-size and full-precision; the student is half the size and running in MXFP4. Because a teacher's output distribution is architecture-agnostic, the size mismatch doesn't block the transfer. The student never sees hard labels, only the teacher's output distribution, matched through KL divergence on the logits.
The Numbers: 4-Bit Beats 16-Bit
Applied to GPT-OSS 120B → 60B, the QAH model (MXFP4) beats the recovered 60B bfloat16 checkpoint on 7 of 9 benchmarks:
| Benchmark | 60B BF16 | 60B MXFP4 (QAH) | Delta |
|---|---|---|---|
| AA-LCR (long-context reasoning) | 35.3 | 42.7 | +7.4 |
| AIME 2025 (math) | 70.7 | 76.3 | +5.6 |
| Aider (agentic coding) | 38.2 | 40.9 | +2.7 |
| τ²-bench (tool use) | 59.4 | 61.7 | +2.3 |
| GPQA Diamond (science) | 65.7 | 67.4 | +1.7 |
| IFBench (instruction following) | 58.4 | 59.9 | +1.5 |
| LiveCodeBench (coding) | 65.5 | 66.5 | +1.0 |
| MMLU-Pro (knowledge) | 74.0 | 73.8 | −0.2 |
| SciCode (science coding) | 35.6 | 34.2 | −1.4 |
The two losses are under a point and a half. The biggest gains land exactly where compression usually hurts most: long-context reasoning and math. The 4-bit model also beats the full-size 120B teacher on LiveCodeBench (66.5 vs. 66.0) and comes within 1.6 points on GPQA Diamond.
QAH vs. QAT: Stability Is the Deployment Difference
To isolate the loss function's effect, they quantized GPT-OSS 9B to MXFP4 and tracked average performance across MMLU-Pro, LiveCodeBench, and GPQA Diamond. Both methods reach a similar peak — 54.9 for QAH vs. 54.6 for QAT — so on best-case accuracy they're effectively tied. The difference is in the curve:
- QAH peaks in ~100 steps, about 7× faster than QAT's 700 steps, then holds.
- QAT collapses sharply past its peak, shedding nearly 19 points by step 1,200.
That's a real deployment risk difference. A QAT checkpoint needs careful early stopping against a held-out signal to avoid shipping a model that's already degrading. A sufficiently trained QAH checkpoint can be served safely because KL distillation against a frozen teacher gives the student no incentive to drift once it matches.
The Catch
One sharp-eyed commenter on the post flagged a legitimate method gap: the headline table compares checkpoints with unequal training. The bf16 60B got one distillation pass; the MXFP4 60B got that pass plus a second one against the 120B teacher. The authors conceded the point — the table doesn't isolate "distilling under quantization" from "distilling for longer". They frame QAH as an applied case study, not a theoretical claim: the shipped 4-bit checkpoint scores above the bf16 it came from, which is the usual outcome of direct post-training quantization. The causal reading that "quantization is free training" needs a proper control arm, which they flag as a limitation.
For a deployment team, though, the practical question is narrower: given a compressed model that has to ship in 4 bits, what's the best way to run the quantization step? QAH answers that. At 4-bit precision the model uses roughly 4× less weight memory than bfloat16, and at half the parameter count it roughly halves compute per token. If you ship in bfloat16 rather than 4-bit, the combined reduction would be closer to 8× less compute per token. With long-context healing up to 32k tokens handled via chunked KL divergence, the training budget stays inside fixed GPU memory.
I'd watch whether the authors publish the missing control arms — the identical second distillation pass in bf16, and post-training-quantizing that extended checkpoint. Until then, treat the efficiency story as solid and the accuracy gains as real but possibly inflated by longer training. Still, the direction is clear: quantization doesn't have to be a tax you pay for efficiency.
Sources: