Context Compaction: When Summaries Lie to the Agent

In Brief

Compaction now has three footings: a server-side API feature that Anthropic’s docs call the recommended strategy (in beta, 150,000-token default trigger, with the SDK tool runner as the older path), an RL training objective in the pipeline behind the 744B-A40B GLM-5.2, and a measured failure surface.
The techniques sort on one axis, reversibility: reversible reduction keeps a pointer to what it drops, lossy summarization is where context silently disappears, and learned compression trains the model to decide what to keep.
A July belief-divergence study found that lossy harness transforms often preserve task success while shifting the agent’s own read on risk and failure mode; it did not test window summarization directly, so the transfer to compaction is plausible rather than proven.
Swapping only the summarizer moved CompactionRL’s pass@1 by 6.5 absolute points on a random 200-task SWE-bench Verified subset, which makes the summarizer a first-class determinant of success rather than a utility, and Cognition says it has fine-tuned a smaller model for the job.
The practitioner ordering: prefer raw over reversible reduction over lossy summarization, compact for quality before the window fills, keep the recent tail raw, and instrument the agent’s beliefs as well as whether it finished.

Why This Matters

Any agent that runs long enough will overflow its context window mid-task. The mechanism teams reach for, compress the history and keep going, now ships as a server-side API feature, in beta and recommended by the vendor’s own docs. Whether a compaction pass drops the one observation that mattered, or quietly rewrites the agent’s model of the task while the benchmark stays green, is a reliability decision teams are making today, mostly by default. Knowing which reductions are reversible, and what to instrument, separates a durable long-horizon agent from one that confidently finishes the wrong job.

Key Findings

Figure	What it measures	Source
150,000 input tokens	Default trigger for Anthropic’s server-side compaction (strategy `compact_20260112`, beta header `compact-2026-01-12`), the docs’ recommended strategy	Claude compaction docs
100,000 tokens	Default `context_token_threshold` for automatic compaction in Anthropic’s SDK tool runner, the older client-side path	Claude compaction cookbook
744B-A40B	GLM-5.2, the open model whose RL training pipeline includes CompactionRL; CompactionRL rounds the size to 750B	Z.ai GLM-5 README
6.5 points (49.0% to 55.5%)	Change in pass@1 on a random 200-task SWE-bench Verified subset, mean of two runs, from swapping only the summary agent, execution agent held fixed at GLM-4.7-Flash	CompactionRL, Table 1
over 95%	Share of the gpt-4.1 teacher’s performance retained by ACON’s distilled compressors	ACON
26–54%	Reduction in peak token usage from ACON’s method relative to running uncompressed	ACON
89 tasks, 80 million tokens	Single-session run with no measurable degradation under structured eviction, on Terminal-Bench 2.0	Beyond Compaction
18 models	Models evaluated in Chroma’s Context Rot report, with performance degrading non-uniformly as input length grew	Context Rot

Compaction is now a primitive, a training objective, and a measured risk

A year ago, “compact the context” meant a coding agent summarizing its own history when it ran low on room. That behavior has since hardened into infrastructure on three fronts, and the spread is what earns the topic a practitioner’s attention now.

As an API feature, compaction now runs server-side. Anthropic’s compaction docs call it “the recommended strategy for managing context in long-running conversations and agentic workflows,” and it is in beta: requests carry the compact-2026-01-12 beta header, the strategy is compact_20260112, and the default trigger is 150,000 input tokens. The older, client-side path is the SDK tool runner’s compaction_control block, per the Claude compaction cookbook (November 2025), which takes an explicit context_token_threshold (default 100,000 tokens) and an optional dedicated summarization model.

On the SDK path the mechanism is blunt. When the threshold trips, the runner injects a summary request and Claude emits a summary. Then, in the cookbook’s own description, “the entire conversation history (including all tool results) is replaced with just the summary.” The retained-versus-discarded ledger is explicit: ticket IDs, categories, and progress status survive, while “full knowledge base article text, complete drafted response text, detailed classification reasoning” are discarded. That is not a lossless operation, and the docs do not pretend otherwise.

As a training objective, compaction has moved inside the model. CompactionRL (Li, Hou, Jing, Tang, Dong) trains the agent to jointly optimize task execution and the summaries that let it continue. The authors state plainly that “CompactionRL is thus deployed in the RL pipeline for training the open GLM-5.2 model (750B-A40B).” Compaction no longer happens only in the harness; a frontier model is now explicitly trained to do it to itself.

As a measured risk, the harness-induced belief-divergence study (Yi, Song; July 2026) supplies instrumented evidence that lossy context-shaping can change what the agent believes, not just what it can recall. That finding gets its own section below.

Lilian Weng’s July 4 harness-engineering roundup frames the surrounding discipline’s baseline rule the same way: a harness “should not carry the entire workflow and all logs in context; instead, it should keep durable state in files.” Compaction is the operation that happens when you can’t, or didn’t.

The reason this matters earlier than window overflow suggests: models degrade well before the window is full.

Chroma’s Context Rot report (Hong, Troynikov, Huber; July 2025) evaluated 18 models spanning several generations, GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 among them. It found that “model performance degrades as input length increases, often in surprising and non-uniform ways”: not a smooth slope but an uneven decline. The useful window is smaller than the advertised one. Chroma’s tasks were deliberately simple and controlled to isolate input length, though, and it never compared compacted against uncompacted agent runs — so compacting before the limit is a hypothesis its results motivate rather than one they establish, and whether summarization loss outweighs the rot it avoids is a per-workload measurement.

The design space has one real axis: reversibility

Strip away vendor vocabulary and the compaction techniques sort by a single question: can you get the dropped information back?

The compaction design space sorted by reversibility: reversible reduction keeps a recovery path to what it drops, lossy summarization and its trained variants discard the originals, and structured eviction rejects the summarize-and-discard primitive altogether. The axis is adapted from Manus’s restorable-compression principle; the four-family grouping is editorial.

Reversible reduction (offloading). The safest family drops only what can be reconstructed and keeps a pointer to it. Manus’s context-engineering writeup (Yichao “Peak” Ji, July 2025) is the clearest statement of the principle: “Our compression strategies are always designed to be restorable. For instance, the content of a web page can be dropped from the context as long as the URL is preserved,” and, bluntly, “any irreversible compression carries risk.”

Lance Martin’s writeup of a Manus webinar (October 2025) sharpens the operational form. Every tool result carries a full format (raw content, parked in the sandbox filesystem) and a compact format (a file-path reference), so the agent can refetch on demand. The file system becomes external memory; the context holds addresses, not payloads.

Lossy summarization. In production coding agents, the default “compaction” is irreversible: summarize the trajectory, discard the originals, resume. Anthropic’s context-engineering guide (September 2025) defines it as “the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary,” and describes Claude Code’s version as preserving “architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs or messages.” This is where the risk lives, and Anthropic names it directly (see the next section). LangChain’s context-engineering taxonomy (July 2025) files this under “compress” — one of four strategies alongside write, select, and isolate — and flags the catch: “summarization can be a challenge if specific events or decisions need to be captured.”

Learned compression. The frontier response to lossy summarization’s fragility is to stop hand-writing the summarizer and train it. Two approaches bracket the space.

ACON (Kang et al., Microsoft/KAIST, October 2025) optimizes a compression prompt by analyzing failed trajectories in natural language, then distills the large-model compressor into a small one, reporting that “distilled compressors retain over 95% of the performance of the gpt-4.1 teacher” while cutting overhead. Separately, ACON’s overall method cuts peak token usage by 26–54% relative to running uncompressed, while improving task success over existing compression baselines.

CompactionRL goes further and folds the summary into the RL objective itself, so the model learns summaries that maximize downstream task success rather than summary readability. Both start from the same admission ACON states outright: “token truncation or generic summarization are insufficient, as they easily discard these critical details essential for multi-step reasoning.”

Structured eviction. A minority position argues the whole summarize-and-discard move is the wrong primitive. “Beyond Compaction: Structured Context Eviction” (Semenov, Dorofeev; May 2026) annotates the trajectory as “typed, dependency-linked episodes” and evicts content by a deterministic, LLM-free policy. It drops the oldest-and-most-recoverable items by dependency rather than oldest-in-time.

Its pitch is the list of failures it claims to avoid relative to summarization: “unpredictable lossiness, destruction of causal structure, blocking model cost, and compression-induced hallucination,” alongside a striking single-session result: “89 sequential tasks across 80 million tokens with no measurable degradation in task accuracy relative to per-task isolated sessions.” That is Terminal-Bench 2.0’s full 89-task suite run as one continuous session instead of the standard per-task isolated protocol. It is one team’s self-reported result for a protocol it invented — a single continuous session per run, with the Terminal-Bench numbers averaged across five runs — and should be read as a promising direction rather than a settled win. The critique it levels at summarization is the honest one, though.

The deeper compaction failure is rewritten beliefs, and a pass/fail benchmark cannot see it

The teams running compaction in production keep hitting the same first-order failure: you cannot know in advance what will matter. Anthropic puts it crisply: “overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later.” Manus reaches the identical conclusion from production: “you can’t reliably predict which observation might become critical ten steps later.” This is why the reversible-reduction camp keeps pointers instead of trusting a summarizer’s judgment about relevance.

The more consequential failure is subtler than dropped facts, and it is what the belief-divergence study measures. The authors elicit a structured K-step belief rollout from the agent: its own read on progress, risk, recoverability, constraints, failure mode, uncertainty, success probability, repair cost, and next action. They then compare how that trajectory evolves under different harness treatments while holding the task, environment, and base model fixed.

The tested treatments are harness transforms, not window-summarization per se. But two of them, compressed repairs and evidence pruning, are compaction by another name, which is why the central finding lands for anyone running it: “blocked actions, compressed repairs, selective verification, and cost-aware evidence pruning often preserve terminal success while changing the beliefs that drive later decisions.”

When a repair-heavy harness compresses a failure-and-fix into a clean repaired transition, the study finds transition compression to be the dominant information loss in that setting: expanding those compressed repair traces materially changes the agent’s beliefs even as the final outcome holds. What it does not establish is which belief is the better one — the authors are explicit that a divergence on its own implies neither better nor worse task performance, and that their metrics do not establish environment-grounded correctness. The conclusion the authors draw is one the field has mostly ignored: “harness design is an experimental variable in agent evaluation, not an implementation detail.”

Put the two together and “when summaries lie” stops being merely rhetorical, as a transfer rather than a proven equivalence.

The study tested compressed repairs and evidence pruning, not window-overflow summaries directly. But a compaction pass is the same kind of lossy transform, so the mechanism plausibly carries: it can leave the agent able to finish the task while quietly changing the beliefs it is acting on. How risky the task is, what has already gone wrong, what it should do next: a pass/fail benchmark, by construction, cannot detect any of that shifting.

This is confidence flattening at the architecture level: the compressed transcript reads as settled fact, and the uncertainty that lived in the raw failure path is gone.

The summarizer decides task success, which is why teams are training dedicated ones

If the summary is where information and belief get rewritten, then the summarizer’s quality decides everything downstream, and the evidence that it dominates is quantitative. CompactionRL reports that changing only the summary agent, holding the execution agent fixed at GLM-4.7-Flash, moves pass@1 by 6.5 absolute points (49.0% to 55.5% in their Table 1) — on a random 200-task subset of SWE-bench Verified, averaged over two evaluation runs. That makes the summarizer a first-class determinant of task success rather than a utility bolted onto the agent.

The production teams reached this conclusion before the RL papers did. Cognition’s Don’t Build Multi-Agents (Walden Yan, June 2025) describes introducing “a new LLM model whose key purpose is to compress a history of actions & conversation into key details, events, and decisions,” and is candid that “this is hard to get right” — hard enough that “you might even consider fine-tuning a smaller model (this is in fact something we’ve done at Cognition).”

ACON’s distillation and CompactionRL’s joint training are the same instinct formalized: the compressor deserves its own optimization loop. For most teams the practical read is that off-the-shelf “summarize this conversation” prompting is one of the weakest points in a long-horizon agent, and the one most worth investing in, whether by a purpose-built prompt, a distilled model, or careful schema design.

There is a real cost tension here that the sources don’t resolve. A dedicated summarization pass is a blocking model call on the critical path (“Beyond Compaction” lists “blocking model cost” among summarization’s drawbacks), and a fine-tuned compaction model is infrastructure most teams won’t build. The reversible-reduction camp sidesteps both by not summarizing at all until it has to, which is the ordering the implications below make concrete.

Compaction alone does not carry a multi-session agent

One more limit belongs here, because it is the failure teams hit right after they get single-window compaction working. Anthropic’s long-running-agents guide (November 2025) reports that even Opus 4.5 across multiple context windows falls short on compaction alone (“however, compaction isn’t sufficient”) because “compaction doesn’t always pass perfectly clear instructions to the next agent.”

The two failure modes they observed are specific and recognizable. Over-ambition: an agent takes on too much, runs out of context mid-implementation, and leaves the next instance undocumented work. Premature completion: a later instance sees existing progress and wrongly declares the project done.

Their fix is external scaffolding rather than a better summarizer: explicit feature lists, progress files, and git commits that survive independent of any summary. Compaction manages the window; durable artifacts manage the horizon. Teams that treat compaction as the whole context-management story tend to discover this boundary the hard way, one hallucinated “task complete” at a time.

Practical Implications

Order your reductions: raw > reversible reduction > lossy summarization. The ordering is about recoverability, not raw performance. Chroma’s context rot means more raw tokens can themselves cost quality, so “keep raw” holds only while the window is still serving you. When raw stops paying its way, drop only what you can reconstruct and leave a pointer (file path, URL, artifact ID) the agent can refetch from: the Manus pattern. Reach for irreversible summarization only when reversible reduction stops freeing enough space. Staged this way, the lossy step touches only the oldest slice of the trajectory.
Compact for quality as well as capacity. Degradation sets in well before the window is full (Chroma’s context-rot finding), so the trigger threshold is a quality knob as much as an overflow guard. If long-horizon accuracy matters more than token cost, test compacting earlier against running raw — Chroma establishes the rot, not that the summary is the cheaper loss — and keep the active window comfortably under the advertised limit. Treat the trigger threshold (trigger on the server-side strategy, context_token_threshold in the tool runner, or their equivalent elsewhere) as a tunable you set against observed task degradation, not a default you leave and forget.
Never summarize the recent tail. Keep the last few steps in raw, full-detail form; CompactionRL retains the two most recent steps by default. The immediate next decision depends on precise recent state. Compress the distant past, not the live edge.
Externalize memory: addresses in context, payloads on disk. Writing durable state to files and holding references in the window (Anthropic’s note-taking pattern, Manus’s file-system-as-memory, Weng’s “keep durable state in files”) turns most of the compaction problem into a retrieval problem, which is reversible by construction.
Treat the summarizer as a component you own. A generic “summarize the conversation” prompt is one of the weakest links in a long-horizon agent and moves task success by several points on its own. Invest proportionally: design a schema for what the summary must preserve (goal, completed actions, open errors, current state, next steps; the CompactionRL summary contract is a reasonable template). If the domain justifies it, distill or fine-tune a dedicated compaction model rather than borrowing the main model’s off-the-shelf summarization.
Instrument beliefs as well as outcomes. A passing benchmark can hide a compaction pass that rewrote the agent’s model of risk and failure. Where reliability matters, periodically elicit the agent’s own read on progress, remaining risk, and known failures, and check that it survives compaction intact: the belief-rollout idea applied as a smoke test. If the agent’s stated confidence or failure-mode read jumps discontinuously across a compaction boundary, treat that as a prompt to check the summary against grounded outcomes: the divergence flags that something changed, not which side of the boundary is right.
Back compaction with durable artifacts for anything spanning sessions. For multi-window or multi-session work, don’t rely on the summary to carry intent to the next instance. Maintain an external, human-legible record of scope, done-vs-remaining, and decisions (feature list, progress file, commit history). Compaction keeps a single run inside its window; artifacts keep a long project from declaring itself finished early.

Open Questions

Is summarize-and-discard the right primitive at all, or a transitional one? Structured eviction argues the causal graph of the trajectory should govern what’s dropped, not a summarizer’s prose judgment, and reports no measurable degradation across 80 million tokens in one session. Whether that generalizes beyond one team’s self-reported result on one suite, and whether dependency-annotation overhead is worth it versus a well-tuned reversible-reduction scheme, is unsettled.
What is the actual cost curve of good compaction? A dedicated summarization model is a blocking call on the critical path, and a fine-tuned compactor is infrastructure most teams won’t build. Public reporting on the latency and dollar cost of production compaction, and where the break-even sits against simply buying a larger context window, remains thin.
How much does training-time compaction (CompactionRL) transfer to harnesses it wasn’t trained against? A model trained to compact its own context inside one agent framework may or may not compact well under a different harness’s triggers, summary schema, and tail-retention policy. Whether learned compaction is a portable capability or couples the model to its training harness is unresolved in the public results.
Can belief drift be caught cheaply in production? The belief-rollout diagnostic is a research instrument; eliciting a nine-dimensional belief vector on every compaction is not free. Whether a lightweight proxy, a single confidence or failure-mode probe across the boundary, is enough to flag the dangerous cases is an open and practically important question.
Where does the security surface move when compaction becomes automatic? A summarizer is a model reading a full untrusted trajectory and deciding what the next instance sees. If attacker-influenced content can steer what a compaction pass preserves or drops, compaction becomes a place to plant or erase context across a trust boundary. The analysis of that surface hasn’t caught up to the technique’s adoption.

Scout: Context Compaction for Long-Horizon Agents — Techniques, Tradeoffs, and When Summaries Lie