HydraFusion's Cascade and Critique: Routing in the Harness

In Brief

GitHub’s HydraFusion routes each request to a single, cascade, or critique workflow and reports 67% lower cost with +4.9 points on TerminalBench 2.1 against a Claude Opus 5 baseline, but its other two benchmarks came in slightly below the baseline, and the model pool and gate criteria are undisclosed.
Handing a strong model the cheap model’s full trajectory recovered under half the quality gap on SWE-bench Verified at roughly 4–6× the cheap model’s cost, while dropping the trajectory and keeping the repository edits recovered 64–84% of it and still cost more than starting on the strong model, per the Handoff Tax paper.
A cascade’s quality gate has a blind spot that grows with the drafting model’s capability, and every metric read through the gate inherits it: one study’s cascade dashboard read 3% error against 13–14% delivered error with a fixed drafter, and against 32% once a retraining loop had degraded the drafter, measured on math tasks rather than code.
A critique step is only as independent as the critic’s context: prior scores passed as metadata anchored seven of eight judge models, and same-family judges show a 3.4–8.4 point lift, which is the case for GitHub’s cross-family, read-only critic.
The two routers that account for the prompt cache both avoid switching mid-thread: LangChain’s picks a model once at thread start and lists mid-thread re-routing as future work, and the Harness Tokenomics router switches only at session start, subagent launch, or side lanes; HydraFusion’s cascade escalates after a draft, which is the handoff the Handoff Tax measures.

Why This Matters

For two years the cost lever in agentic coding was which model to call. The harness has since become a second lever, and routing between execution patterns is its newest form. GitHub offers it as a Copilot research preview; LangChain and an enterprise-economics paper each describe a router built into the harness with reported savings. Underneath all three sit two questions a vendor post cannot answer: what a strong model should inherit when a cheap draft is rejected, and whether the gate that rejects it can be trusted. Both were measured in late August and September 2026. The answers decide whether a draft-and-gate cascade is a cost saving or a quietly degraded product.

Key Findings

Routing between execution patterns is now a vendor preview with a benchmark attached

Project HydraFusion, a research preview GitHub published on September 4, 2026, picks one of three workflows per request. In GitHub’s words, single means “One selected model solves the task directly.” Cascade means “An efficient model drafts a solution and a quality gate decides whether to accept it or escalate to a stronger model.” Critique means “One model drafts a result, an independent read-only critic from a different model family reviews it (following the same review pattern as Rubber Duck), and the drafting model revises once.”

The router “evaluates each request and chooses the least complex workflow expected to meet its needs, using additional model calls only when they are likely to improve the result.” It reads “capability signals for reasoning, code generation, debugging, and tool use.” That, plus a line that HydraFusion “treats workflow selection as an optimization problem,” is the whole public description of the decision logic. The gate’s acceptance criteria and the models in the runtime pool are not stated. On what the stronger model receives, GitHub says solver steps “use the shared workspace,” and its documentation warns that a discarded draft’s file edits “aren’t undone automatically”; whether any of the draft’s transcript travels with the escalation is not described.

HydraFusion’s three execution patterns and the two undisclosed points inside them, the gate’s criteria and what the escalated model inherits beyond the shared workspace; adapted from GitHub’s HydraFusion announcement.

The evaluation is offline and first-party. GitHub measured “verified task quality, which is the share of tasks confirmed as correctly answered, and the complete estimated workflow cost,” against Claude Opus 5 and GPT-5.6 Sol baselines. The published deltas are against Opus 5.

Benchmark	Cost vs. Opus 5	Quality vs. Opus 5	Source
TerminalBench 2.1	67% lower	+4.9 points	GitHub
DeepSWE	36% lower	−1.5 points	GitHub
CheckpointBench	65% lower	−0.1 points	GitHub

Read the table as a whole. On two of three benchmarks the routed system was a little worse than the single strong model, at a third to two-thirds the cost. That is a cost result with quality held near parity, and a real one. The headline row suggests something more than that.

GitHub also scopes the preview: “For this preview, first-turn, single-prompt coding tasks are the best place to start.” Single-prompt tasks are where a draft-and-gate loop is easiest to evaluate, because the gate judges one artifact rather than a conversation.

Pricing follows the models used: “Usage is based on the tokens consumed by the models HydraFusion uses, priced at each model’s standard rate.” A cascade that escalates therefore pays for the draft, the gate, and the stronger model. On September 30 GitHub extended the preview to VS Code 1.140 and the Copilot app, where it “remains a research preview and is subject to change” and Business and Enterprise administrators must enable preview features.

Early users want to see the routing. In the community discussion under the launch, one user wrote “I don’t have any insights on what model is doing,” another asked whether the router respects an enterprise’s enabled-model list, and a third objected that it was selecting GPT-5.4, “both worse and ten times more expensive than luna.” Another called the “complete lack of transparency and control” the biggest gripe with auto-routing features. One user reported credit consumption near a tenth of what they were used to. A GitHub staff member acknowledged the visibility feedback.

Stephen W. Thomas’s walkthrough saw a cascade run take three to four minutes and 25 credits on one prompt, which he labels an observation rather than a benchmark. The opacity is before the run, not after it. GitHub’s documentation says the CLI shows the chosen pattern and each pass as it runs and keeps a summary of the steps afterwards, VS Code and the Copilot app show the models used when a user hovers over a completed response, /usage reports the credits each model consumed, and /collect-debug-logs archives “a detailed record of the execution pattern and the models used for each step.” What a user cannot do is predict the route or the bill before the request runs, or see the list of models the router draws from, which GitHub says is not fixed.

The escalated model should inherit the repository, not the transcript

The cascade pattern has an unstated design decision at its core. When the gate rejects a cheap draft, what does the stronger model see? The Handoff Tax (Roy Ganz, Mor Shpigel Nacson, Adi Kalyanpur, Ron Litman; August 25, 2026) measured the options.

The authors paired low-cost and high-cost models (LC and HC, in the paper’s terms) within two families, Claude Haiku 4.5 with Opus 4.7 and GPT-5.6 Luna with Sol, on the 500 tasks of SWE-bench Verified. They varied handoff direction, timing, and interface: full-trajectory transfer, compaction, and trajectory removal with the repository state preserved.

The headline: “Across both model families, full-trajectory escalation recovers less than half of the LC-to-HC quality gap while incurring a substantial cost premium.” The per-interface figures in the paper body are the useful part.

Escalation interface	Claude pair, gap recovered	GPT pair, gap recovered	Source
Raw full trajectory	47%	36%	Handoff Tax
Compaction	60%	40%	Handoff Tax
Trajectory dropped, repo edits kept	64%	84%	Handoff Tax

The cost side is worse than the quality side. “Raw escalation substantially increases cost over LC-only in both families, by approximately 4.0× for Claude and 6.1× for GPT. For Claude, this increase is large enough that Raw costs more than twice as much as starting with HC ($1.61 vs. $0.72).” A cascade that escalates with the full transcript can cost more than never cascading at all, and recover less than half the quality it was meant to buy.

Trajectory removal does not escape the premium. In the paper’s setup it still cost more than starting on the strong model, $0.81 against $0.72 for Claude and $0.50 against $0.47 for GPT, and scored below it, 72.4% against 79.2% and 79.7% against 83.7%. The paper escalates every task at fixed switch points rather than only the drafts a gate rejects, so a real cascade’s bill depends on how often the gate accepts.

Direction reverses the advice. Downshifting from the strong model to the cheap one after the hard reasoning is done, with the full trajectory, raised the Claude pair’s pass rate from the cheap model’s 54.6% to 65.6% while cost rose from $0.41 to $0.51.

Prefix compaction did slightly better in that pair, recovering 56% of the gap against raw transfer’s 50%, while the GPT pair’s best downshift was the raw trajectory at 79%. Dropping the strong model’s trajectory on a downshift was the lowest-recovery downshift interface in both families, at 28% of the gap in the Claude pair and 53% in the GPT pair.

The authors’ summary: “In escalation, reducing LC trajectory information improves the trade-off: compaction favors savings, whereas trajectory removal favors quality recovery. In downshift, removing the HC trajectory while preserving its working-tree edits reduces quality across both model families.” Timing follows the same asymmetry in the Claude pair: later escalation generally recovers less, later downshift more.

The handoff interfaces with the highest measured quality recovery on SWE-bench Verified: dropping the transcript on escalation in both families, and keeping it on downshift, raw in the GPT pair and compacted in the Claude pair; adapted from Ganz, Shpigel Nacson, Kalyanpur, and Litman, The Handoff Tax.

A second paper appears to contradict this and does not. Handoff Debt (Dipesh KC, Anjila Budathoki; revised August 30, 2026) interrupts a coding agent at deterministic points, freezes the repository, and hands the task to a successor under four views: repository only, raw trace, summary notes, structured notes. Across 75 source tasks, 181 handoff points, and three successor models, “context-bearing handoffs reduce median agent events by 20-59% and cumulative prompt tokens by 42-63% relative to repository-only takeover.” The raw trace cut median events 57–59%, more than either note format, for all three successors.

The two studies measure different handoffs. Handoff Debt’s predecessor is a Qwen agent and its successors are Qwen, Gemma, and Devstral, so the receiver is a peer, not a stronger model, and the outcome is efficiency. The paper itself says “Solved-rate effects are smaller and model-dependent, but efficiency gains are consistent.” The Handoff Tax measures quality recovery when the receiver is the stronger model.

Put together: a peer resuming work should get the trace, because rediscovery is the cost. A stronger model fixing a failed draft should get the edits and a clean prompt, because the cheap model’s reasoning is what failed. A cascade is the second case.

Every cascade routes on a verdict, and the verdict comes from a model. Cheap Verifiers, Large Blind Spots (Dushyant Rajput, Nirdesh Chauhan, Siddharth Kosaraju; September 1, 2026) measured the verifier’s blind spot, “the fraction of the student’s wrong answers it accepts.” It “grows with student capability (β from 0.12 to 0.55 as the student scales 0.5B to 32B) and shrinks with verifier capability, so it is worst in the cheap-student, cheap-verifier regime cascades exist to create.” The better the drafting model, the more plausible its wrong answers look to a cheap gate.

Buying a stronger gate returns the saving. In the same study a frontier verifier “drives β to about 0.05 but then escalates on 46% of hard-MATH queries against a 39% true error rate, paying the frontier price on nearly half of all traffic.”

The fourth finding is the one for anyone reading a vendor’s cost chart: “the cascade’s own dashboard, every metric computed through the verifier, reads a flat 3% error while true delivered error swings up to 32%: the system is blind to its own degradation by construction.” The 32% is the endpoint of the paper’s self-improving variant, which fine-tuned a Qwen2.5-7B student on the verifier’s rejections each round and degraded it; the frozen student in the same run delivered 13–14% error against the same 3% dashboard, which is the gap a fixed cascade carries. These are math tasks on small open models, not coding agents on frontier models, so the magnitudes do not transfer. The mechanism does, and GitHub does not say how its benchmark tasks were “confirmed as correctly answered,” so its “verified task quality” metric is only as good as whatever verified it.

Stacking gates does not escape the problem. Partially Correlated Verifier Cascades (Jiangang Han; July 15, 2026) models verifiers that share the generator’s blind spots and finds reliability saturates below one: “once ρv is large, adding gates buys almost nothing — reliability is bought by decorrelating the verifier from the generator (different model family, different modality, external oracles, tool-based checks).” In the note’s worked regime, independence assumptions understate the failure rate 20× at five gates and about 3,000× at ten.

The note is theory with a synthetic check, and it lists those levers without ranking them. For a coding harness the list still reads as a ranking: a test run is a better gate than a second model, and a second model from another family is a better gate than one from the same family.

The accepting path is where the savings live, which is why acceptance rate matters more than escalation cost. RLM-Cascade (Haifeng Wu and colleagues; June 22, 2026), a proxy that drafts with a cheap model and verifies with Opus, reports a draft-use rate of 88.8% across 125 Claude Code production requests, a 45.8% cost cut against direct Opus, and a faster median because the no-verify path dominated. Its quality check is a 20-task benchmark, so treat the 100% versus 95% pass figure as a smoke test.

OpenRouter’s Jev-verified cascade cookbook shows the pattern’s accounting plainly: the frontier model is re-run on the original question without seeing the draft, only two of 50 questions escalated, and “Every tier is paid.” Its own controls are the useful part: the cascade shipped zero wrong answers at $0.012, the frontier model alone shipped two wrong at $0.175, and the cheap model alone shipped zero wrong at $0.004, so on that smoke test the cascade bought nothing the cheap model did not already deliver at a third of the price.

A critique step buys independence only if the critic’s context is engineered for it

HydraFusion’s critique pattern inherits its design from Rubber Duck, GitHub’s April 2026 cross-family reviewer. The rationale there: “a model reviewing its own work is still bounded by its own training biases: the same training data and techniques, the same blind spots.” GitHub’s evaluation on SWE-Bench Pro found “Claude Sonnet 4.6 paired with Rubber Duck running GPT-5.4 achieved a resolution rate approaching Claude Opus 4.6 running alone, closing 74.7% of the performance gap between Sonnet and Opus.” That is a first-party number on one benchmark, and the clearest first-party evidence that a cheap drafter plus a cross-family critic approaches a strong single model.

Two findings explain why the critic’s isolation is the design choice to get right. Anchoring Bias in LLM-as-a-Judge Systems (Ante Kapetanovic, Kemal Altwlkany, Andro Mercep, Tomislav Duricic, Emanuel Lacic; August 26, 2026) passed judges a prior score as metadata and found that such scores “anchor judgments and systematically shift ratings toward their values.” Seven of eight judges showed the effect across 185,271 successful evaluations, with Cohen’s d reaching 0.71; the eight spanned GPT-4.1, Claude Sonnet 4.5, DeepSeek-R1, and open-weight Llama, Qwen, and Gemma models.

On categorical industry data with human labels, anchored metadata “blocks 48% of error corrections and flips 10.18% of correct judgments toward an assigned wrong label.” Neither chain-of-thought nor a warning to disregard the metadata removed the effect. Code review was the weakest and most heterogeneous of the four task categories: its mean shift was −0.047 on the 0–5 scale against −0.28 to −0.34 for creative writing, factual QA, and summarization, and the paper’s model-based estimate for code review at temperature zero, −0.067, carries a 95% interval of [−0.141, 0.011] that spans zero. The authors’ safeguard is structural: “Excluding experimentally irrelevant metadata remains the direct safeguard when workflow design permits it.”

For a critique loop, the metadata the paper tested maps onto a gate score or an attempt count that travels with the artifact. A drafter’s own confidence, a draft that reports itself complete, was not tested; reading the result onto it extends the paper rather than quoting it. HydraFusion’s critic is “read-only” and tool-less, and GitHub’s documentation says assisting models “receive only the context they need”; whether that context includes the drafter’s self-assessment is not stated.

Kiro’s Workflows, released September 30, make the isolation explicit: “Each step runs in its own session with fresh context, so a reviewer, for example, evaluates the work without inheriting the coder’s reasoning.” Its bundled pipeline runs one review step on Claude Opus 5.5 and another on GPT-5.6 Sol.

Family matters as well as context. Who Judges Matters (David Ababio Awuni and colleagues; September 15, 2026) ran a fully crossed pairwise design across four open-weight families and, after correcting for candidate quality, found “a positive same-family lift (3.4-8.4 percentage points)” in every family. A same-family critic is a correlated gate in the Han sense. Direction matters too: which model reviews which is not symmetric, and a reviewer pointed the wrong way can cost more than it recovers.

GitHub’s choice of a cross-family, read-only critic is consistent with all three results. The open item is the critic’s input, which GitHub has not documented beyond that one line.

Routers that account for the prompt cache switch at boundaries

The two routers that price the prompt cache both avoid mid-thread changes today, for reasons that only partly overlap. RLM-Cascade’s rule-based router, by contrast, picks a path per agentic turn. LangChain’s router for Open SWE (Sydney Runkle and Eugene Yurtsev; October 1, 2026) classifies the thread’s first human message into a fast, balanced, or performance tier (GLM-5.3-Flash, GPT-5.6 Sol, GPT-6 Astra) and keeps that model for the thread, a design the authors call naive and a proof of concept; subagents currently pick their model independently of it. Across 973 threads, 34% went fast, 56% balanced, and 10% performance.

Median cost per task fell 64%, from $2.61 to $0.94; the mean fell 42% and the 90th percentile 37%. Routed threads produced merged pull requests 29.2% of the time against 27.3% for the always-Astra control, a difference the authors report as not significant (p = 0.49).

Mid-thread re-routing sits on the post’s future-work list with its trade-off stated: “The cost is the prompt cache: switching models throws it away, so the new model re-reads the thread at full price.” The authors add that “for async agents that cost is often negligible,” because a short cache TTL expires between human turns anyway, so the penalty falls on sessions whose turns arrive inside the cache’s lifetime. That is the Handoff Tax’s cost premium from the billing side. The post’s placement argument is the harness thesis in one sentence: “Choosing the right model requires domain and task context that the harness already assembles, and a gateway typically lacks.”

Harness Tokenomics (Ted Kwartler, Alan Aqrawi, Arian Abbasi; September 24, 2026) reaches the same architecture from enterprise economics. “One user turn is many requests over a prompt cache that belongs to one model, so the router moves work only where no running conversation has to rebuild its cache. It routes at session start, in side lanes and at subagent launch.”

Repricing about 10,000 real sessions, the authors find a crossover where the highest-priced model costs less than the next tier on long tool-heavy sessions. In an emulated 10,000-seat enterprise the router “recovers 13 to 21% of model spend.” The number is modelled, not observed. It is also far below HydraFusion’s headline, because it counts whole enterprises rather than first-turn tasks.

The cheapest routing decision may not be a model switch at all. Anthropic’s cost guidance argues for effort calibration first: “Claude Fable 5.1 at low effort matches the performance of Fable 5 at high effort at a third of the cost.” On its FrontierCode Diamond figures, Fable 5 scores 11.5% at low effort for $5.35 a task and 30.9% at max effort for $19.00. A router that can turn an effort dial inside one model avoids the handoff entirely. It keeps the cache only where the model allows it: the same post says effort and thinking settings “render into the prompt ahead of your content, so they are part of the cached prefix,” and names Opus 5 and Fable 5.1 as the models on which effort can change mid-conversation without breaking it.

All of this is a harness result in the sense Sydney Lewis’s Same Model, Different Harness (August 26, 2026) established the same week as the Handoff Tax. Changing only context management under a 20,480-token window on 169 SWE-bench Verified tasks raised mean fail-to-pass from 28% to 49% and complete solutions from 43 to 72, with the frozen treatment transferring to three other models.

Lewis concludes that evaluations “should treat the model and harness together as the tested solver.” A router is one more harness component under that rule, and the case for benchmarking harness and model as a pair extends to the router that chooses between them.

Build or wait: the classifier is being absorbed, the gate and the handoff policy are not

The routing decision is becoming a commodity. Cloudflare’s Clef and Clef-flash, released October 1 under Apache 2.0, are 27B and 9B decision models that return probabilities over typed answers at 209.3 ms and 38.8 ms median latency, trained with a cross-entropy loss plus a Brier term for calibration and compatible with the Jev API. OpenAI previewed a Decisions API at DevDay that gives its Luna model “a predefined set of options to choose from,” per Simon Willison’s live blog, with no schema or pricing published.

LangChain’s router already runs on Jev, which it says made classification almost 50× faster. The classifier that picks a tier is, at Clef-flash’s size, a sub-50-millisecond call anyone can rent.

What is not being absorbed is everything around the classifier. The gate’s blind spot is a property of your drafter-verifier pairing on your tasks, and the mitigation the cascade papers offer is decorrelation through tests, tools, or a different family. The handoff interface is a policy you write, and the obvious default, raw transcript transfer, measured worst on escalation. The critic’s context is a prompt-assembly decision.

HydraFusion documents almost none of this for its own system: a shared workspace on escalation, a critic given “only the context they need,” and nothing on the gate. That is a reasonable reading of how early the product is, and a reason not to treat its 67% as a number you will see. It is the same split the absorption argument reached for the harness as a whole: the part that governs survives, the part that compensates gets eaten.

Practical Implications

Decide the handoff interface before the router. If a cascade escalates, give the stronger model the repository edits and a fresh prompt, not the cheap model’s transcript. The Handoff Tax measured raw transfer at under half the quality gap recovered and up to 6.1× the cheap model’s cost; trajectory removal recovered 64–84%, and in the paper’s escalate-every-task setup still cost more than starting strong, so the saving rests on the gate’s acceptance rate. Reserve trajectory transfer, raw or compacted, for downshifts, where dropping it measured worst in both families.
Gate with something the drafter cannot fool. A test run, a build, or a type check is a decorrelated verifier; a second call to the same family is not. Where a model gate is unavoidable, use a different family and measure its blind spot on a held-out set of known-bad drafts, because the cascade’s own metrics will not show it.
Give the critic a clean room. Strip gate scores and attempt counts from the critic’s context, per the anchoring paper’s safeguard, and treat the drafter’s self-assessment the same way, which the paper did not test. Code review was the paper’s weakest category, so measure the effect on your own critic. Kiro’s fresh-session-per-step is the shape to copy. Pick the critic’s family and direction deliberately rather than by default.
Route at boundaries you already have. Thread start is where LangChain’s router switches, and session start, subagent launch, and side lanes are where the Harness Tokenomics router does; the prompt cache is the cost both name for switching mid-thread. Before adding a model switch, test whether an effort dial on one model gets the saving without a handoff, on a model whose cache survives the change.
Treat vendor routing as a cost preview, not a quality guarantee. HydraFusion’s two near-parity benchmarks and its first-turn scoping describe what it does today. Keep a shadow evaluation outside the router that checks delivered quality against an unrouted baseline, since any quality figure a cascade computes through its own gate inherits the gate’s blind spot.
Rent the classifier, own the policy. Clef, Jev, and the Decisions API make the tier decision cheap and fast. The handoff rule, the gate, and the critic’s context are where the measured outcomes diverge, and none of them ship in a decision model.

Open Questions

Does trajectory removal on escalation hold for multi-turn sessions? The Handoff Tax measured single-task SWE-bench runs and GitHub scopes HydraFusion to first-turn prompts; a rejected draft three turns into a conversation has state that a fresh prompt may not reconstruct.
What is the gate blind spot for frontier coding drafters? The 0.12-to-0.55 range comes from small open models on math, and neither cascade study measures β for a Sonnet-class drafter under a GPT-class gate on repository tasks, which is the regime vendor cascades actually run.
Does the critic see the gate? HydraFusion’s critic is read-only and cross-family, but whether it receives the drafter’s confidence, a gate score, or the attempt count is undocumented, and the anchoring result says that detail could decide the pattern’s value.
Is a 64% median saving a per-task artefact? LangChain’s and GitHub’s largest gains are measured per thread or per prompt; the enterprise-level model recovers 13–21%. Which figure a team sees likely depends on how much of its traffic is long, tool-heavy sessions, where the Tokenomics paper finds the pricey model can be cheaper.
Will the routing trace become an audit record? GitHub’s CLI shows the pattern and each pass as they run and can archive a per-step record of the models used in its debug logs, and VS Code shows the models on hover; whether that record follows a change into code review, so a team can say which model touched what, is a governance question as much as a billing one.

Scout: Cascade or Critique: Routing Between Execution Patterns Inside the Harness