Chapter 37: Tiered Compression

"Compress the old stuff" — but a file you read forty steps ago is still the file you're editing now.

Quick Track (Master the core in 5 minutes)

1. Age is a proxy for irrelevance, not a measure of it — and sometimes the proxy is wrong
2. On first processing, three tiers classify by distance from the tail: full, semantic summary, metadata stub
3. In the generic compressor, content-bearing tools get a floor; producer-specific aging can still apply separately
4. Compression itself costs model calls; cap them per pass or it becomes a hidden cascade
5. Every rewrite must keep tool-call and tool-result paired, or the provider rejects the turn

Public-source snapshot: implementation details are examples from public Kocoro origin/main at commit 4ec6772, reviewed 2026-07-27. Treat the invariant as the lesson and re-check constants against current source.

37.1 Starting with the File It Was Still Editing

An agent is thirty iterations into a refactor. Early on — iteration 3 — it ran file_read on internal/auth/session.go, the file the whole task is about. That result is now twenty-seven tool-result-bearing messages back.

The first context-pressure pass looks at it and sees an old tool result. Uniform age-based compression strips it to metadata:

[file_read: internal/auth/session.go — 412 lines, 18.2 KB]

Correct by the rule. Catastrophic in effect. The model no longer has the file it is editing. So it reads it again — 18 KB back into the prompt, all the tokens you just saved, plus a round-trip. And in a few more iterations, that read ages out too, and it happens again.

The rule wasn't wrong about age. It was wrong about what age means. Age is a proxy for "unlikely to be needed," and for most tool output that proxy holds. For the file you're working in, it doesn't hold at all.

37.2 Three Tiers, by Distance from the Tail

The structure is a ladder measured by how many later tool-result-bearing messages follow each result:

Tier 3 — the newest 8 tool-result messages stay untouched. This is working memory. The model is actively reasoning over it and touching it costs more than it saves.

Tier 2 — distance 8 through 19 gets compressed but keeps substance. A result over 2,000 characters gets a semantic summary from a model call; below that, or when no summarizer is available, it falls back to mechanical head-and-tail truncation. Either way the content survives in reduced form.

Tier 1 — distance 20 and beyond drops to metadata only. Tool name, arguments, size. Enough to know what happened, not enough to reconstruct it.

Compressed Tier 2 output caps at 300 characters. That's aggressive, and deliberately so: at that distance the result's job is to remind the model what it learned, not to re-teach it.

One implementation detail changes how to read this ladder. For native tool_result blocks, it is a first-touch classification, not a conveyor belt. Once a result is processed at Tier 2, the runtime marks it CompressedTier=2; later passes keep those bytes unchanged even after its distance crosses 20. Tier 1 therefore means “an uncompressed result first encountered at distance 20+,” not “every result eventually becomes metadata.”

At first classification, notice the shape of the degradation. It isn't linear. Recent content is untouched, middle-aged content loses form but keeps meaning, old content keeps only its shape. What's being preserved changes at each step, not just how much.

37.3 The Floor That Fixes §37.1

Tier 1 is right for most tools and wrong for a specific class: the ones where the content is the reason you called them.

Within this generic compression pass, file_read, grep, glob, directory_list, and anything matching browser_* get a Tier-2 floor. If an uncompressed result is first encountered at distance 20+, it degrades to head-and-tail truncation instead of falling through to a metadata stub. The file from iteration 3 keeps its opening and closing lines after that pass.

The reasoning is that a metadata stub for these tools destroys the exact thing that made the call worth making. [grep: "handleRequest" — 47 matches] tells you a search happened. It tells you nothing about what matched, which is the only reason anyone ran a grep.

Browser tools belong in that set for the same reason and it's worth spelling out: an actionable page snapshot IS the task payload. An agent working through a web flow needs to know what was on the page, not merely that a page was observed.

The floor is not a global retention promise. During normal request assembly, the producer-specific browser/GUI observation window in Chapter 45 runs separately and replaces text observations older than the newest three with one-line stubs. The Tier-2 floor protects content from the generic pressure compressor; the observation policy owns when an obsolete viewport stops being actionable.

The generalizable test: would a metadata stub of this result let the model continue, or force it to re-run the tool? If re-running is the only option, you didn't compress — you deferred the cost and added a round-trip.

37.4 Compression Is Not Free

Tier 2's semantic summaries are model calls. Compress twelve results in one pass and you have made twelve extra model calls to save context — which can easily cost more than the context was worth.

So the number of semantic compressions per pass is capped at 2. Everything else in the Tier 2 band falls back to mechanical head-and-tail, which costs nothing. The cap is on attempts, not successes, so a failing summarizer can't burn the budget silently and then leave you uncompressed anyway.

There's a second exemption running alongside: some tools skip micro-compaction entirely, on the same logic as the Tier-2 floor. Summarizing a page snapshot into prose loses the structure the model was going to navigate.

The rule worth carrying: compression that costs model calls needs a budget, and the budget should be on attempts. Otherwise a bad day for the summarizer turns your context optimization into a cost multiplier.

37.5 Every Rewrite Can Break the Turn

Both tiers rewrite message content in place, and there are two ways that goes wrong.

Pairing. A tool_use block and its matching tool_result are a unit. Rewrite one without the other — or drop a result whose call is still present — and providers reject the request outright. Which is why compression needs a map from tool_use_id to tool name and arguments before it starts: it has to know what each result was in order to write a metadata stub for it, and it has to leave the pair intact.

Cache. Every in-place rewrite invalidates the prompt cache from that message forward. That's unavoidable — the whole point is changing the content — but it must be attributable. Each rewrite emits a cache-compaction event tagged with which tier did it, so when cache hit rates fall you can tell whether it was Tier 1, Tier 2, or something else entirely.

Uninstrumented rewrites are the ones that make cache debugging impossible. You see the hit rate drop and have no way to find which of a dozen prompt-shaping mechanisms did it. Chapter 39 is the full discipline.

37.6 Snapshot Evidence

These describe one dated implementation, not a universal contract.

37.7 When Tiering Is the Wrong Model

Age-based tiering assumes relevance decays monotonically. When it doesn't, the model breaks.

A long-running comparison task may need result #2 and result #40 side by side at the end. A debugging session may return to an early stack trace after twenty steps of exploration. In both cases the newest-is-most-relevant assumption is simply false, and no floor rescues you — the floor preserves head-and-tail, not the middle.

If that's your workload, the answer isn't a better tier boundary. It's offloading: spill to disk and let the model retrieve deliberately, rather than guessing what it will want from position in the transcript.

Also: don't tier a short session. Below the Tier 3 window there is nothing to compress, and running the machinery anyway just adds a code path that can fail.

37.8 Common Pitfalls

No floor for content-bearing tools. §37.1 — the model re-reads, and you paid tokens and a round-trip to save tokens.

Uncapped semantic compression. Twelve summary calls to save context that was worth less than twelve summary calls.

Capping successes instead of attempts. A failing summarizer retries forever and never trips the budget.

Breaking tool-call pairing. The provider rejects the whole request, and the error points at a message index rather than at your compression code.

Uninstrumented rewrites. Cache hit rate drops and nothing in your telemetry says which mechanism did it.

Rewriting compressed content. A 300-character Tier 2 summary does not need another pass, and changing it again would dirty the cache prefix. Track what has already been reduced.

Key Points

1. Age is a proxy for irrelevance, and proxies fail. The file you're editing is old and still essential.
2. Classify in kind, not just in volume. On first processing choose full, meaning-without-form, or shape-only, then keep rewritten bytes stable.
3. Content-bearing tools need a floor. A metadata stub of a grep destroys the reason the grep existed.
4. Budget the compression itself, on attempts. Otherwise saving context becomes a cost multiplier.
5. Preserve pairing and instrument every rewrite. One breaks the request; the other breaks your ability to debug the cache.

Next: Chapter 38 moves from result size to schema size — the other half of what fills a prompt.