Q8_0 Is 5× Closer to BF16 Than FP8. The Log-Probs Can't Tell.

A while back I found that Qwen 3.8 Flash-Next scored far worse on vLLM than on llama.cpp for a reason that had nothing to do with the weights: vLLM’s Anthropic bridge was handing the model its own prior reasoning back, and it degenerated into repeating one stock sentence (the full autopsy). Fixing the transport took vLLM from 64.3 to 74.0 against the llama.cpp anchors’ 75.7 mean.

That left 1.7 points. One run against three, comfortably inside the anchors’ 73.0–78.4 band, so possibly nothing at all. But the two stacks were also serving different weights — the official FP8 checkpoint on vLLM, Unsloth’s Q8_0 GGUF on llama.cpp — and nobody had measured how far apart those actually are.

So I measured all of them against the official BF16 release: 76,488 tensors, 179.6 billion parameters, 61 minutes on CPU, one tensor at a time in host RAM.

The weights really are different, and not in the way I expected

Where both formats quantize — routed experts and the PLE table — Q8_0 sits about 5× closer to BF16: 0.54% relative RMS error against 2.66%, which is 45 dB of signal-to-quantization noise against 31.5 dB. That is not a marginal difference. It is the gap between two formats that both call themselves 8-bit.

But the chart’s real shape is the hollow markers. FP8 stores everything except the routed experts, the PLE table and MTP as exact BF16 — attention, GDN, shared experts, hyper-connections, embeddings, the LM head. The GGUF quantizes nearly all of it at ~0.6%. The two formats are not better and worse. They are differently wrong.

The arithmetic decides which matters. Routed experts and the PLE table are 172 of 179.6 billion parameters — 96% of the model. FP8 is 5× worse on that 96% and perfect on the rest, so on any weighted view of the whole model, Q8_0 is the more faithful checkpoint by a wide margin.

The two errors have different characters, too, and it is worth knowing which you are buying:

• FP8’s error is e4m3’s 3-bit mantissa, not its scale layout. It lands at 2.66% almost everywhere, tensor after tensor, with a worst case of 2.67% — extremely uniform. It flushes almost nothing to zero (0.001%).
• Q8_0’s error is absmax block scaling and is set by outliers: a single large value in a 32-element block coarsens the step for the other 31. Its median is far lower, but it flushes 0.73% of non-zero entries to zero against FP8’s 0.002%, because values more than ~127× below their block’s maximum have nowhere to go.

On the PLE table — one logical tensor of 51.2 billion entries — that shows up starkly. FP8 uses a single global scale for the whole table and rounds 59% of entries into the 1e-4…1e-3 error band. Q8_0 uses a scale per 32 entries and puts 71% of its error an order of magnitude lower, at 1e-5…1e-4, while zeroing 374 million small entries that FP8 keeps.

Three checkpoints to get Q8-grade experts into vLLM

vLLM cannot load a GGUF for this architecture, so the fidelity advantage was not available where I wanted to serve. I built three INT8 checkpoints in the AutoRound/GPTQ layout the patched vLLM already loads, with a new Q8_0-packed disk-backed PLE:

C is the interesting one: Q8_0’s expert fidelity and FP8’s exact dense tensors. A second full tensor-by-tensor run — the same 76,488 tensors, 70 minutes — confirms it did what it was supposed to:

• Its routed experts and PLE table are bit-identical to the GGUF’s. Not equivalent — identical, 24,576 of 24,576 tensors exact in each expert class, and the table exact in one of one. The repack preserves the Unsloth bytes exactly.
• Every dense class it inherits from B is exact BF16: shared experts, attention, GDN, hyper-connections, embeddings, the LM head, the PLE projections. Where the GGUF sits at 0.53–0.66%, C sits at zero.
• MTP is exact BF16, against FP8’s 2.66% and the GGUF’s absence.

So C matches whichever shipped checkpoint is closer to BF16 in every class, and beats both in at least one. That shows up in the worst case too: the single worst tensor anywhere in C is an expert at 0.814%, where the GGUF’s worst is a GDN tensor at 1.028% — a tensor C stores exactly.

One verification worth reporting because it could so easily have gone the other way. I quantized the PLE table from the BF16 reference using ggml’s own rule, and the result is byte-identical to Unsloth’s. Not close — identical. All three checkpoints therefore share one table, and Unsloth’s conversion of this tensor is exactly reproducible from the official weights.

Getting there required inverting a set of transforms the GGUF converter applies that are documented nowhere I could find: every norm except one stored as w + 1, the GDN block’s 48 value heads re-ordered by a stride-16 permutation across six different tensors, ssm_a stored as −exp(A_log), and conv1d reshaped. The expectation going in was that there were none. Every comparison above is in the HF domain with those transforms undone.

And then the measurement couldn’t see any of it

With four checkpoints serving — FP8, A, B, C — I scored the per-token log-probabilities of a 23,729-token corpus through each.

The ordering is exactly what the weight fidelity predicts: the two checkpoints carrying the GGUF’s experts are best, RTN-128 next, FP8 last. It is also meaningless at this sample size, and the control says so.

Serving the identical FP8 checkpoint on two different pipeline layouts, its top-1 token agreed with itself only 95.55% of the time, with a mean absolute log-prob difference of 0.095. That is the measurement’s own reproducibility floor — the same weights, the same software, differing only in how layers were split across cards. Cross-checkpoint agreement ran 94.4–95.5% at 0.099–0.120. The two ranges overlap. One pair of different checkpoints (C against B) agreed more closely than one pair of identical ones.

So a 5× difference in weight error produces an output difference that this corpus cannot distinguish from pipeline-layout noise. The total NLL spread across every checkpoint is 0.006, which is 0.6% of perplexity.

That is the finding I would carry away from this, and it generalizes past this model: a log-prob or perplexity comparison on a 20k-token corpus is not a sensitive enough instrument to separate 8-bit quantization formats, even ones that differ five-fold in weight fidelity. If you have picked a quant on that basis, you may have measured nothing. The judgment eval — hundreds of decisions scored against ratified ground truth, where the earlier transport bug showed up as a 10-point gap — is the instrument with a demonstrated ability to resolve differences this size. Log-probs are not.

Serving, for completeness

All four cards in a four-rank pipeline (19/6/6/17 layers) beats the old three-rank layout by about 8% on prefill, leaves 4–10 GB free per rank, and is the only layout that fits the group-32 checkpoints with multi-token prediction enabled at all — at three ranks, A missed its KV allocation by 30 MiB.

Prefill at 8k / 32k / 131k tokens: FP8 5,541 / 6,026 / 6,120 t/s · A 5,429 / 5,882 / 6,001 · B 5,518 / 5,886 / 5,939 · C 5,392 / 5,549 / 5,931. INT8 Marlin runs 2–3% behind FP8 Marlin on these cards; both are weight-only W8A16. Decode (91–119 t/s) tracks MTP acceptance, which varied with the random prompt more than with the checkpoint.

Two operational notes that cost time. vLLM’s eager safetensors loader with 4.3 GB shards across four workers swapped 28 GB on a 31 GB host and stalled the load outright; the lazy mmap loader is required. And the group-32 checkpoints carry 7.5 GB of scales against FP8’s one scale per 128×128 block, which is most of why they are 17 GB larger.

C is in production: four cards, MTP k=3, vision on, native 262,144 context, a KV cache of 442,611 tokens, 8 of 8 smoke tests including a 250k-token prompt at 5,003 t/s prefill.

What is still open

C is the best-fidelity checkpoint available for this model on vLLM — measured, not assumed, in every tensor class — and it costs 2–3% of prefill against FP8. That much is settled. Whether any of it moves the score is not: the log-prob measurement returned a null, and the only instrument that has ever resolved a difference this small on this pipeline is a six-hour eval run. Until those runs exist, the honest statement is that the weights were 5× different, the outputs were not measurably different, and the 1.7 points remain unexplained.