Harness Absorption: What Models Have Eaten, What Survives

In Brief

The absorption claim has a real record: agentic reasoning loops and context compaction have moved from external scaffolding into trained model behavior, per OpenAI’s own accounts; Anthropic reports newer models need far fewer explicit behavioral rules — consistent with absorption, though its post never says how the judgment was acquired.
The headline exhibit — Anthropic cutting over 80% of Claude Code’s system prompt — is partly relocation into tool descriptions, skills, and memory rather than pure deletion, by the authoring engineer’s own description.
Harness choice still drove a 23.8-point aggregate gap in a controlled six-harness comparison averaged over eight model backends — and up to 11.5 points at a single fixed model — with sensitivity concentrated in the cheaper, weaker models cost-conscious teams route work to.
Three research programs published or updated in August 2026 — HELIX, StateM, Self-Harness — treat the harness as a model-specific, regenerable object; the direction they suggest is automated harness engineering, not its disappearance.
Drew Breunig argues Fable’s premium price ended the era when the next model arrived cheaper and papered over harness neglect; the multi-model routing that follows is exactly the regime where harness quality matters most.

Why This Matters

An infrastructure team deciding this quarter’s investment faces two credible stories about the same layer. Dan McAteer says the model keeps eating the harness, so anything built to compensate for a model weakness gets deleted within a generation or two. The harness-sensitivity benchmarks say harness choice still decides tens of points in aggregate, and double digits at a single fixed model. Both are true at once. The resolution is an accounting question: which harness components depreciate on the model release cycle, which hold value across it, and what the write-off schedule should be.

Key Findings

Absorption has a real record: reasoning loops, compaction, and behavioral rules have moved into weights

McAteer’s essay states the thesis in one line: “Models keep absorbing the harness into their weights — soon, it will be a harness for human attention rather than for the model”. His history runs from ReAct in late 2022, when the agent loop existed only as prompt text, through the failed autonomy bets of AutoGPT and BabyAGI, to the present co-training era. In that era, reinforcement learning runs inside vendor harnesses, and models internalize what the scaffolding did for them. Engineers then delete the absorbed parts.

The history explains why the claim keeps coming true on schedule. AutoGPT bet on autonomy in spring 2023 and lost to arithmetic: at 95% per-step reliability, a 20-step task succeeds about 36% of the time, per McAteer’s accounting. The industry retreated to human-in-the-loop IDEs. Then reasoning models crossed the reliability threshold, and terminal agents like Claude Code met the moment with a comparatively thin harness. Each stage ended the same way: the scaffolding that compensated for one generation’s weakness became dead weight for the next.

The migrations he names have public accounts behind them, though the accounts vary in strength. Context compaction is the cleanest case. Harnesses long implemented it as external summarize-and-truncate scripts. Then OpenAI billed GPT-5.1-Codex-Max, released in November 2025, as its first model natively trained to work across multiple context windows through compaction — the capability moved from wrapper code into training, per OpenAI’s announcement and MarkTechPost’s coverage.

The ARC-AGI-3 episode shows what it costs when a harness lags the moving boundary, though the boundary here is the vendor’s API rather than the weights. OpenAI reported that the benchmark’s official harness discarded the model’s private reasoning after every action and deleted its oldest history when the window filled. Enabling retained reasoning and compaction — two existing API settings — took GPT-5.6 Sol from 13.3% to 38.3% on the public set, with roughly six times fewer output tokens.

Read that as a boundary story rather than a weights story. Both settings are Responses API context management — per OpenAI, passing the previous-response ID retains reasoning across turns, and compaction replaces rolling truncation — so memory management had migrated into the vendor’s platform, not into the model. The episode does not show a capability trained into weights. It shows the cost of mismatch: the generic scaffolding was discarding state the platform would have preserved, and the external version was no longer merely redundant. It actively suppressed the vendor-managed one.

Three reported migrations of harness functionality into model behavior, each ending with the scaffolding deleted — the first two documented as training changes, the third only as newer models needing fewer explicit rules; adapted from Dan McAteer, Latent Space.

The headline shrinkage is relocation as much as deletion

The system-prompt exhibit deserves a closer read than the absorption argument gives it. Anthropic removed over 80% of Claude Code’s system prompt for the Claude 5 generation with, per the company, no measurable loss on coding evaluations. But Thariq Shihipar’s own write-up describes much of the cut as a move, not a disappearance: “we found we could delete these repeat examples and put instructions on how to use tools in the tool descriptions rather than the system prompt.”

The post frames six shifts. Only one reads as absorption — rules become judgment, because “newer models have better judgement and can handle these decisions well without explicit rules” — though the post reports the disappearing need for rules without saying where or how that judgment was acquired.

The other five are interface redesign: examples retired because “giving examples actually constrains them to a certain exploration space,” with tool and file design taking their place; instructions consolidated into tool definitions; context loaded progressively through skills instead of upfront; manual memory files replaced by auto-memory; and markdown specs replaced by richer references like code and test suites.

For the depreciation question, the split is the finding. Prompt text deleted because the model internalized the behavior is a depreciated asset. Instruction mass moved closer to its point of use is a refactoring — the harness changed shape, and the engineering that designed the tools, skills, and memory surfaces retained its value. Reading the full 80% as absorption overstates the case the number gets cited for.

There is also a reading of the six shifts that supports McAteer’s endpoint. Progressive disclosure, auto-memory, and richer references are all attention management: deciding what the model sees, and when. The prompt shrank, but the design work moved up a level, from writing instructions to shaping the information surface. That already resembles an interface for attention — it just rations the model’s attention rather than the human’s.

Harness choice still moves aggregate scores by 20+ points, fixed-model scores by 10+ — and sensitivity concentrates off the frontier

Whatever the trend line, the current level is high. The measurements below vary the harness or its configuration. Most hold the model fixed within a comparison; the headline gap averages over an eight-backend pool, and the cost row deliberately varies the model.

Measurement	Result	Source
Best vs. worst of six harnesses, same task set and model-backend pool, 106 tasks	76.2 vs. 52.4 — a 23.8-point gap	Harness-Bench
Cross-harness spread on Claude Opus 4.6 vs. on Qwen3.6-35B-A3B	2.3 points vs. 11.5 points	Tongyi Lab
GPT-5.6 Sol on ARC-AGI-3, official harness vs. two API settings	13.3% → 38.3%	OpenAI
Terminal-Bench 2.1 via harness scaling, GPT-5.6 Sol xhigh, no weight changes	95.3%	StateM
StateM run cost, DeepSeek-V4 Flash vs. GPT reference configuration	$52.22 total ($15 final run) vs. $574.68	StateM
Self-improved harness vs. initial harness, nine model-benchmark pairs	relative gains up to 132%	Self-Harness

Harness-Bench, the May 2026 benchmark from Yao, Tan, Liu, and colleagues, ran six configurable harnesses against eight model backends over 106 sandboxed tasks: “NanoBot obtains the highest aggregate score (76.2), while OpenClaw obtains the lowest score (52.4), giving a 23.8-point gap under the same task set and model-backend pool.”

For scale, one model generation buys 8.8 raw-accuracy points in StateM’s reference numbers — GPT-5.5 xhigh at 83.1% to GPT-5.6 Sol Ultra at 91.9% on Terminal-Bench 2.1. The two figures are not directly comparable: Harness-Bench’s aggregate combines completion, security, and process signals on its own task suite, so the harness gap and the generation gap are measured in different units, on different tasks. The case for benchmarking harness and model as a pair rather than the model alone rests on the size of the Harness-Bench gap, not on a cross-benchmark subtraction.

The distribution of that sensitivity is the newer finding. Harness-Bench reports that stronger backends score higher with lower cross-harness variance, while weaker backends swing more — their performance is more dependent on the surrounding execution substrate. Tongyi Lab’s PawBench analysis of 4,050 model-harness-task combinations puts numbers on it. Claude Opus 4.6 varied 2.3 points across three production harnesses while Qwen3.6-35B-A3B varied 11.5. And a weaker model on the right harness (qwen3.6-plus on QwenPaw, 76.5) beat a stronger one on the wrong harness (qwen3.6-max-preview on Hermes, 70.2).

StateM pushes the same lever to its limit. By reorganizing execution around durable state, phase-local context, checked transitions, and recoverable runbooks — no weight changes — it reports 95.3% on Terminal-Bench 2.1 with GPT-5.6 Sol xhigh, lifts GPT-5.5 xhigh from an 83.1% baseline to 92.1%, and gets a DeepSeek-V4 Flash configuration to frontier-comparable results for $52.22 of total spend — under $38 of adaptation plus roughly $15 of final-run API usage — against $574.68 for the GPT reference run.

A caveat on reading StateM’s headline: the 95.3% and the $52.22 belong to different runs. The frontier score costs frontier money, and the cheap run is a cheaper model reaching results the authors call comparable. Nor is it a controlled same-harness comparison — the runs share the StateM runtime, runbook structure, and golden rules, with the under-$38 adaptation tuning the DeepSeek configuration on top. What transfers cheaply is the harness architecture, which is the point.

Put the two halves together and the absorption thesis gets a scope restriction. Absorption is measured at the frontier, where vendors co-train models with their own harnesses. Sensitivity is largest below the frontier — precisely the models teams route to for cost. The harness is depreciating fastest where fewest production tokens will run.

Research now treats the harness as co-evolving and regenerable, not dissolving

The August research wave frames the harness as a first-class object with its own scaling behavior, not residue awaiting absorption. HELIX (Fan and Huang, August 2026) argues the coupling between the two layers motivates co-evolving them rather than optimizing either alone: “The harness shapes both what a model can accomplish and the trajectories from which it learns.” Its proposal is a loop — evolve harnesses against a fixed model, train the model on the verified trajectories the harness produced, rebuild the harness for the improved model. Absorption, in this framing, is one arc of a cycle that immediately regenerates new scaffolding.

Self-Harness (Zhang and colleagues, June 2026, revised August 20) automates the rebuild. An agent mines its own execution traces for model-specific failure patterns, proposes minimal harness modifications, and validates them — no human engineer in the loop. Across nine model-benchmark combinations spanning Terminal-Bench 2.0, SWE-bench Verified, and AppWorld, on MiniMax M2.5, Qwen3.5-35B-A3B, and GLM-5, the abstract reports relative gains of up to 132%.

The premise behind that automation deserves its own sentence: harness design is inherently model-specific, because different models fail differently. That is why hand-building one harness per model stopped scaling, and it is also why a harness tuned for last quarter’s model quietly underperforms on this quarter’s.

Notice what the two camps actually agree on. McAteer’s absorption thesis and the co-evolution papers both hold that a harness is specific to a model generation and short-lived. The disagreement is about who rebuilds it: McAteer expects vendor RL to absorb each layer and leave less harness behind, while HELIX and Self-Harness expect tooling to regenerate a new model-specific harness each cycle. Under the first story harness engineering shrinks; under the second it becomes something you automate rather than something you stop doing.

Fable’s price rewrote the depreciation schedule

The economics of neglecting the harness ran on an assumption Drew Breunig names in “Fable & The End of the Free Lunch”, quoted by Simon Willison: “A new model would arrive at the same price (or cheaper!) and paper over most of your problems.” Harness investment felt wasteful because its returns were about to be given away for free.

Fable broke the pattern by being, in Breunig’s account, incredible and expensive at once. He puts GLM 5.2 at roughly 1/9th of Fable’s cost and about 1/5th of Opus 5’s. His recommended response is routing: interrogate designs with the premium model, delegate routine coding to cheaper models with great context. “So we started to think about what work went where.”

The release cadence sets the depreciation clock for whatever you build. Per ARK Investment Management data built on Artificial Analysis figures, reported by OfficeChai, the industry median gap between frontier releases across five major US labs fell to 11 days for 2026 year-to-date, per an April 28, 2026 read of the data. The per-lab medians in the same read were 49 days for OpenAI and 71.5 for Anthropic.

A harness component built to compensate for a specific model weakness therefore has a useful life of one release at the lab you depend on — now measured in weeks, not years. That schedule is inferred from release cadence, not measured: none of the studies here tracks a harness component across model generations to see when it actually stops earning its keep. The scaffolding layer also churns far faster than weights within a single generation, which is why regression attribution defaults to blaming the wrong layer.

But note the tension inside Breunig’s own prescription. Routing work to GLM-class models to escape Fable’s price moves production tokens exactly where the sensitivity data says harness quality decides the most — an 11.5-point spread, not a 2.3-point one. The same economics that shortens the life of frontier-compensating scaffolding raises the return on harness engineering for the cost-optimized fleet. Absorption and rising harness ROI are not in contradiction; they apply to different tiers of the same routing table.

Practical Implications

Keep a depreciation ledger for the harness, by component class. Model-compensating scaffolding (retry wrappers for flaky tool calls, external compaction, verbose behavioral rules) should be written off over one model generation at your primary lab — currently weeks, a schedule set by release cadence, not measured decay. Interface work such as tool definitions, skills, and memory surfaces relocates across generations rather than dying, per the Claude Code restructuring. Evaluation and observability appreciate, because they are what tells you which scaffolding the new model just made deletable. The attention layer of interrupt policy and approval gates is the predicted survivor under McAteer’s thesis.

Harness component classes ordered from shortest useful life (left) to longest (right).

Instrument before you delete. The deletion pass needs regression evidence, and a pass/fail resolve rate cannot supply it. Trajectory-level measurement is what shows whether removed scaffolding was dead weight or quietly doing work.
Schedule a deletion pass with every model upgrade, and treat Anthropic’s six shifts as its checklist. Rules the new model handles as judgment, examples that now constrain exploration more than they guide it, context that can become progressively disclosed — each is candidate scaffolding to remove. The ARC-AGI-3 result is the warning for skipping the pass: stale scaffolding does not just waste tokens, it can suppress context management the vendor’s platform now provides, and by triple-digit relative margins.
Budget more harness engineering, not less, when you adopt multi-model routing. The cheaper tiers of a routing table are the harness-sensitive tiers. A team sending routine work to GLM- or Qwen-class models should benchmark harness-model pairs for that tier specifically, since a cheaper model on a well-fitted harness demonstrably beats a pricier model on a poor one in PawBench’s data, and matches a frontier reference at a fraction of the cost in StateM’s.
Watch the co-evolution tooling before staffing a harness team. If Self-Harness-style automation matures, per-model harness tuning becomes something a pipeline regenerates on each model release rather than something engineers hand-author. The durable skill is then defining the evaluation the tuner optimizes against — which returns to the same conclusion: the evals outlast the scaffolding.

Open Questions

Is the attention layer itself absorbable? McAteer predicts interrupt policy and approval routing survive as the harness’s final form, but nothing in the absorption record says trained models cannot internalize escalation judgment the way they internalized compaction — the governance question is whether anyone should let them.
Does cross-harness variance actually shrink generation over generation? The stronger-models-vary-less finding comes from single snapshots; a longitudinal read across model generations on a fixed harness pool does not yet exist in the sources reviewed here.
Are third-party harnesses structurally disadvantaged by co-training? If vendor models are increasingly RL-trained inside their own harnesses, the fit between an open-source harness and a frontier model may degrade in ways benchmark snapshots understate. Public evidence on this remains thin.
How much of StateM’s harness-scaling gain does the next frontier generation absorb? The paper’s mechanisms — durable state, checked transitions, recoverable runbooks — read like a specification of what vendors will co-train next; the 95.3% figure is either a durable architecture or a to-do list for someone else’s RL pipeline.

Scout: Harness Absorption: Is Harness Engineering a Depreciating Asset?