Chapter 35: Context Compaction

"Just summarize when it gets full" — except the summarizer is a model call too, and it has to fit in the window you already blew.

Quick Track (Master the core in 5 minutes)

1. Three gates, not one: proactive at 0.90, pre-flight at 0.95, reactive on provider error
2. Thresholds come from token headroom arithmetic, never from a round percentage
3. Keep primers + summary + a recent tail with a hard floor; repair tool-pair boundaries
4. Cap the summarizer's own input, and allow exactly one reactive retry — never reset the flag
5. Every rewrite invalidates cache from that point on; derive boundaries from stable inputs

10-Minute Path: 35.1-35.3 -> 35.6 -> Kocoro OSS Lab

Public-source snapshot: implementation details are examples from public Kocoro origin/main at commit 4ec6772, reviewed 2026-07-27. Treat the invariant as the lesson and re-check constants against current source.

35.1 Starting with a Turn That Died at Minute 91

Your agent has been refactoring for 90 minutes. Thirty files read, four test runs, the whole diff plan sitting in context. Iteration 34, it reads one more file:

[iter 34] file_read  path=build/generated/schema.pb.go
          result: 412 KB
[iter 34] → assembling request... 203,417 tokens
[iter 34] ✗ provider: context_length_exceeded

So you compact and retry. Obviously.

But look at where you're standing. You are compacting while already over the limit, and compaction is itself a model call — one that has to fit inside the window you just blew. If the summarizer also fails, there is nothing left to try. You drop the turn, and 90 minutes of accumulated state goes with it.

The 412 KB file didn't kill this run. Having exactly one line of defence did.

35.2 Three Gates, Failing for Three Different Reasons

The fix isn't a better threshold. It's more than one gate — and the reason it works is that they fail independently.

Proactive, at 0.90 of the window, is the normal path: persist learnings, summarize, reshape, continue. It runs on a token estimate, and estimates are approximate. A 412 KB result can sail straight past it between two checks.

Pre-flight, at 0.95, catches exactly that. It re-checks immediately before the request goes out — the last moment when compacting is still cheap instead of desperate.

Reactive fires on the provider's context-length error. It exists because your estimate can be wrong in the direction that hurts, and because the provider is the only real authority on what fits.

Build only the first and you get minute 91. Build only the third and every long session pays for an error round-trip it never needed. The gates are not redundancy — each one is the backstop for a specific way the one above it lies to you.

35.3 Why 0.90, and Not 0.85

Thresholds all look about the same as percentages. Convert them to tokens and the difference shows up.

On a 200K window, 0.85 puts the cliff at 170K and 0.90 puts it at 180K. What lives in that 10K? Every session whose peak lands between 170K and 180K — roughly 5% of them — which now finishes without compacting at all. You didn't just save a summary call. You avoided an irreversible loss of information.

So why not push it to 0.95 and save more? Because compaction has to leave room for the next request and its response. Set the trigger too high and you compact into a window that still can't fit the turn — you paid for the summary and got nothing.

What makes 0.90 usable is that there's a 0.95 gate above it. Take the backstop away and 0.90 stops being aggressive and starts being reckless. Which gives you the rule worth carrying to any system: derive the threshold from the token headroom you actually need, then name what catches the remainder. If you can't name the catcher, your threshold is too high.

This also explains the earlier single-gate harness described in Chapter 32. A lower trigger with only one gate was not wrong — it was a different point on the same curve, chosen before a second gate existed to make a higher trigger safe.

35.4 What You Choose to Lose

Compaction is lossy by definition, so the only real design question is which losses you pick.

Three regions survive. Primers — the opening messages holding the original ask and the system setup. Lose those and the agent will solve a different problem, confidently. Summary — the compressed middle, produced by a model call. Recents — the tail kept verbatim, so "that approach from earlier" still has something to point at.

The public runtime keeps 20 recent turn pairs by default, with a floor of 3 even under budget pressure. The floor matters more than the default. Under pressure the instinct is to keep shaving the tail, but drop below a few turns and the agent loses the thread entirely — it starts re-deriving work it finished twenty minutes ago, which costs more than the tokens you saved.

One detail that will bite you in production: that recent slice is taken positionally from the tail. Cut carelessly and you slice between a tool_use and its matching tool_result, leaving an orphaned half-pair that providers reject outright. Any implementation that keeps a tail needs boundary repair, and it's the kind of bug that only shows up on long sessions.

35.5 The Summarizer Will Also Fail

This is the part most designs skip, and it's where minute 91 actually becomes unrecoverable. GenerateSummary is a model call like any other. It fails, it returns empty, it times out, and on a small tier it overflows on its own input.

So cap that input. The transcript handed to the summarizer is limited to 540,000 characters, head-and-tail with an explicit elision marker in the middle. Skip this and emergency compaction hands a small-tier model something it cannot process — your recovery path dies for exactly the same reason the original request did.

Then bound the retry. The reactive path sets a flag that is never reset for the life of the run:

reactiveCompacted = true // never reset — prevents infinite reactive loops

One emergency retry. If that fails, fail cleanly. An agent that compacts, retries, overflows, and compacts again is spending real money to arrive back where it started.

And record which gate broke. The runtime emits separate phase tags across the proactive, pre-flight, reactive, emergency, and force-stop paths, keeping summary failures distinct from learning-persistence failures — ten of them at this snapshot. At 3am, compaction_failed tells you nothing you can act on. reactive_summary_no_prior tells you which rung snapped and whether there was an earlier summary to fall back to.

35.6 Compaction and the Cache Are Pulling Opposite Directions

Prompt caching pays you for a byte-stable prefix. Compaction rewrites history. Every rewrite invalidates the cache from that point forward — so these two systems are in direct conflict, and the conflict shows up on the invoice.

The public source records what that costs when you get it wrong. An oversized user message was being truncated to a budget derived from the current message slice. As history grew, the cut point moved every single turn. Each turn broke the cache prefix right at the truncated message and re-billed the entire thing as fresh cache creation: roughly $0.67 per follow-up turn, on one message, for no benefit whatsoever.

The fix was to stop deriving the boundary from something that changes. A fixed fraction — 0.80 — of a per-session-stable target, instead of a value computed from however much history happens to exist right now. Identical truncation, stable boundary, cache prefix intact.

That generalizes into something worth writing on the wall: any truncation whose boundary depends on current history size will thrash your cache. Derive boundaries from inputs that don't move. Chapter 39 is the full discipline this belongs to.

Notice the scaling choice too. The fraction applies to the context window rather than a flat character cap — so 200K-era sizing doesn't quietly over-truncate the moment you move to a 1M-context model family.

35.7 Snapshot Evidence

These describe one dated implementation, not a universal contract.

35.8 When Compaction Is the Wrong Tool

Not every context problem is a compaction problem, and reaching for it reflexively costs you quality you didn't need to spend.

If the session will never approach the window, the machinery is pure risk — gate it behind a minimum message count. If the problem is one enormous tool result, spill it to disk behind a pointer instead of summarizing it into the transcript; Chapter 36 is that path, and it should be your first reach, not your second. If the content is the deliverable — a transcript the user asked you to produce — compacting it destroys the product. Compaction shapes working context, never output.

And if the run is audit-bound, compact the model's view while persisting the complete record separately. Compaction is a prompt-shaping operation. It is not a retention policy, and treating it as one will lose you evidence you're required to keep.

35.9 Common Pitfalls

One threshold, no backstop. The most common design in the wild, and the direct cause of minute 91.

Re-compacting the summary. Each pass over already-summarized text loses more signal for fewer tokens saved. Track what's been compacted and stop reprocessing it.

Summarizing with the tier that just overflowed. Convenient, and it fails precisely when you need it most. Route the summarizer to a cheaper tier with its own input cap.

Accepting an empty summary. A summarizer that returns "", taken at face value, silently erases the middle of the conversation. Check for empty, keep the prior summary.

Letting guardrail nudges into the summary input. Last turn's "you appear to be repeating yourself" is not conversation. Strip loop-injected messages first, or your summary will faithfully record that the agent was scolded.

Trusting the estimate. Character-heuristic token counts are wrong at the margin. That's not a flaw to fix — it's the reason pre-flight re-checks and reactive exists at all.

Kocoro OSS Lab (10 minutes)

1. Read window.go and find the comment that derives 0.90 from the 180K cliff. Note that the arithmetic is in the comment, not just the number.
2. Trace the three call sites of compressOldToolResults in loop.go and identify which gate each one belongs to.
3. Write down what your own system would emit instead of compaction_failed — name the five paths that can fail, then check whether your telemetry can distinguish them today.

Key Points

1. Three gates, not one. Proactive on estimate, pre-flight before the call, reactive on provider truth. Each backstops a specific lie the one above it tells.
2. Thresholds come from arithmetic. Derive from token headroom, name what catches the remainder, and write the reasoning into the comment where it can't rot silently.
3. The summarizer is a fallible model call. Cap its input, bound its retry to exactly one, and never reset the flag.
4. Compaction fights the cache. Any boundary derived from current history size thrashes the prefix. Derive boundaries from stable inputs.
5. Reach for offloading first. One huge result belongs on disk behind a pointer, not summarized into your transcript.

Next: Chapter 36 moves large results out of the prompt entirely, and Chapter 37 degrades old results by age instead of summarizing everything at once.