The Amnesia Tax
85% of my agents' tokens were spent rebuilding memory they'd been forced to throw away. One environment variable and a native context window made runs 7× faster — and better.
For days I had a mystery: open-weights models running my eval pipeline consumed 3.1 to 4.2 million output tokens per run and took 22–24 hours, while Claude models did the same work in 170–400k tokens and one to two hours. Same task, same tools, same spec. I blamed the models. I was wrong.
The tax collector was context compaction. My open-weights runs were configured with a 113k context cap, and each run survived 91–108 compaction events — every one of which discards most of the working state, after which the model pays to rebuild it: re-reading files, re-deriving decisions, re-planning. Roughly 85% of a capped run’s tokens were spent recovering knowledge the run had already paid for.
The hidden second ceiling
Lifting the cap exposed a subtler one. My harness is Claude Code, and for models it doesn’t recognize, it silently assumes a 200k context window — which means its auto-compaction fired at ~168k no matter what the server actually offered. The model was serving 262k; the harness was budgeting for 200k; nobody complained anywhere.
The fix is one environment variable:
CLAUDE_CODE_MAX_CONTEXT_TOKENS=262144Verified empirically, not vibes: after setting it, compaction events fired at 229k — 87% of the true window, exactly where the auto-compact margin should put them.
What full native context bought
The first run at a true native window told the whole story: 3.1 hours instead of 23. One compaction instead of a hundred. A ninth of the tokens. Decode throughput didn’t improve — if anything the heavier Q8 quant decoded slightly slower — the entire win was in not repeating work.
And it wasn’t just speed. The flash model’s quality score rose 7.9 points when the cap lifted, and the errors that vanished were omission-shaped — dropped records, forgotten findings — exactly what you’d predict from a worker whose notes get shredded a hundred times a shift. On the 27B, where I changed only the cap and nothing else, the efficiency gains replicated perfectly across three runs; the quality gain there was smaller than run-to-run variance, which is its own lesson.
The general claim
For long-horizon agentic work, context window is the strongest performance lever I’ve measured — stronger than quantization (rungs measured under identical configs differed by 0.4 points), stronger than hardware, stronger than a model generation. The relationship isn’t linear: once the window fits the working set, the compaction-rebuild loop doesn’t shrink, it disappears. My cost curves say a bigger-context model at lower precision beats the reverse every time, and the industry’s benchmark culture — short tasks, fresh contexts — is structurally blind to the effect.
If you run agents against self-hosted models: check what window your harness believes it has. Mine had been lying to itself for days, in production, silently.