Scout: From Prompt to Loop — The Loop Specification and the Software Factory

Summary

Mid-2026 handed agent practice a new unit of work: the loop specification — a bounded, reusable artifact bundling a trigger, a goal, a verification check, a stopping rule, and durable memory, which a human hands to an agent harness in place of step-by-step prompting. The idea has an eighteen-month practitioner lineage running from Anthropic’s and Simon Willison’s “tools in a loop” definitions through Geoffrey Huntley’s brute-force Ralph loop to a June wave in which the head of Claude Code at Anthropic said he no longer prompts Claude at all, Addy Osmani named the discipline “loop engineering,” a public Loop Library catalogued the artifacts, and an arXiv position paper formalized the anatomy. The org-scale extension — the “software factory,” in which pipelines of loops run triage-to-ship with humans supervising — was the recurring frame at the AI Engineer World’s Fair, but its evidentiary base is thin: the concrete numbers are vendors’ own internal dogfooding, the customer trajectories are aspirational, and even a vendor’s deployment lead describes enterprise adoption as stuck among early adopters. The artifact is real and worth adopting now; the factory is a vendor narrative worth watching skeptically — a metaphor making yet another lap through the software industry.

Key Findings

Eighteen months from definition to discipline

The vocabulary consolidated in stages. Anthropic’s December 2024 Building Effective Agents supplied the substrate definition — agents “are systems where LLMs dynamically direct their own processes and tool usage,” typically “just LLMs using tools based on environmental feedback in a loop.” Simon Willison’s September 2025 Designing agentic loops moved the engineering burden outward: an agent “runs tools in a loop to achieve a goal,” and “the art of using them well is to carefully design the tools and loop for them to use.” Geoffrey Huntley’s mid-2025 Ralph technique made the outer loop literal — while :; do cat PROMPT.md | claude-code ; done — with the filesystem (@fix_plan.md, spec files) as memory across iterations and, tellingly, no stopping mechanism at all; Huntley is candid that “you’ll wake up to a broken codebase” some mornings.

June 2026 is when the slogan crystallized. Addy Osmani’s Loop Engineering essay quotes OpenAI’s Peter Steinberger — “You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents” — and Boris Cherny, head of Claude Code at Anthropic: “I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.” Around the same time, Forward Future published the Loop Library, a public catalogue of copyable loops, and a late-June arXiv position paper — Stop Hand-Holding Your Coding Agent, from a solo author at Brazil’s Instituto Federal de Goiás — gave the artifact its name and anatomy. By the AI Engineer World’s Fair days later, it was the conference vocabulary: Latent Space’s dispatch has swyx describing AI engineering’s arc from chat to tools to goals — “We’re all about cron jobs and loops” — with Huntley’s Ralph-loop writing cited onstage as an influence.

Anatomy: what a loop specification concretely contains

The arXiv paper’s definition is the cleanest statement of the artifact: “a bounded, reusable artifact, made of a trigger, a goal, a verification step, a stopping rule and a memory.” The body elaborates five parts — a trigger (“a person, a schedule, or an event”), a goal (“preferably verifiable”), an execution phase in which the agent works “ideally by calling proven, named skills,” a verification that “checks the result for real,” and a stopping rule driving toward a named terminal state (“success, no-op, blocked, stalled, exhausted”) — plus durable memory kept on disk between runs, since the model itself retains nothing from one run to the next.

Three design commitments in the paper do most of the work:

Verification is ranked, and the rank decides autonomy. The paper lays out a five-level verification ladder — deterministic checks (exit codes, assertions, golden outputs), then rules and constraints (linters, schemas, policies), then delayed field truth (deploys, real responses), then model-as-judge scoring, then human checkpoints. Levels 1–2 form what it calls the autonomous zone: the region where a loop can safely run unattended because the check cannot be sweet-talked. It also argues for scoring against a “frozen yardstick” — the same rubric or benchmark every turn — so progress is comparable across iterations. The paper’s sharpest practical claim is that the check, not the instruction, is the hard part of loop design.
Terminal states are named, and failure never masquerades as completion. Distinguishing success from no-op from blocked from stalled from exhausted is what separates a loop specification from Huntley’s run-until-broken bash loop; the paper is explicit that “an error or an exhausted budget never counts as success.”
Memory is durable but curated. State lives on disk — plans, decision records, version history — and the paper warns that lessons accumulated without curation can drag performance below the no-memory baseline, so keep-or-discard judgment on each stored lesson is part of the spec, not an afterthought.

The practitioner artifacts converge on the same fields. The Loop Library’s loop-design guide templates loops as “When [trigger], inspect [fresh inputs]” with an [acceptance check], a [state file], a [budget], a [success gate], a [no-op condition], and an [escalation condition] — and states the lifecycle plainly: “A goal loop starts manually and continues until its check passes or its budget runs out.” Osmani’s building blocks are the same shape at the harness level: scheduled automations for discovery and triage, isolated worktrees so parallel agents don’t collide, skills to write down project knowledge the agent would otherwise guess, connectors into existing tools, sub-agents splitting maker from checker, and persistent state outside any single conversation — a markdown file or a Linear board. Several closely connected practitioner sources landing on trigger-check-budget-state-escalation is the signal that matters: the vocabulary is converging — consolidation within one discourse, not independent replication.

What it is not: a prompt template, the harness’s loop, or an autoresearch recipe

The paper is careful about three boundaries, and they’re the useful ones for practitioners.

Against the prompt template. A loop is, in the paper’s framing, a prompt repeated with scaffolding around it — and the scaffolding is the point. A prompt template parameterizes instructions; a loop specification adds the things a template cannot express: a machine-checkable definition of done, termination semantics with named failure states, a budget, and state that survives the context window. The paper’s justification test is that a loop earns its scaffolding only when the result of one turn changes the next action — otherwise a bare scheduled prompt suffices. Neither replaces the other; loop and prompt are different tools, and the paper explicitly declines to retire prompting.

Against the harness’s internal cycle. Claude Code and Codex already run a perceive-act-observe loop internally — that is the Anthropic definition of an agent. The loop specification sits outside that plumbing: the paper’s metaphor is that the harness supplies the engine while loop engineering writes the pilot. Conflating the two makes the artifact sound like something the vendor already shipped; it isn’t.

Against the autoresearch recipe. “Autoresearch” now names two adjacent things, both distinct from the loop specification. Andrej Karpathy’s autoresearch is a metric hill-climbing experiment loop — modify train.py, run a fixed five-minute experiment, keep or revert on a scalar metric, repeat with no stopping criterion beyond the human returning in the morning. In loop-specification terms it is a specialized instance: verifiable scalar goal, level-1 deterministic check, and a deliberately absent stopping rule. Introspection’s usage — co-founder Roland Gavrilescu, ex-xAI, describes an outer loop where agents improve the primary system itself, captured as reusable “recipes,” with the loop becoming the product — is a self-improvement pattern one level up. The loop specification is the general artifact both cases instantiate: its goal need not be a scalar, its checks span the whole verification ladder, and its stopping rule is mandatory rather than optional. If autoresearch is a recipe for climbing a metric, a loop specification is the contract for any bounded, checkable, resumable unit of agent work.

The corpus check: loops in the wild are more manual and less reusable than the slogan

The paper’s empirical layer is a hand-coding of the Loop Library’s fifty catalogued loops — “we coded the fifty loops of the Loop Library by hand” — and the profile it finds undercuts the automation rhetoric in useful ways. Per the corpus section: “The trigger is manual in seventy-eight percent of loops”; “seventy-eight percent run a single agent”; “Seventy percent sit in the autonomous zone of levels 1 and 2”; only twenty percent “call named, reusable skills”; only thirty-two percent maintain persistent memory; and “human approval appears in thirty-six percent of loops.” The typical published loop, in other words, is a human-started, single-agent job with a deterministic check — closer to a good Makefile target than to a self-running factory cell. The composable-skills and durable-memory parts of the vision are aspirational even inside the catalogue that inspired the paper.

The evidence class matters as much as the numbers. This is a position paper anchored in a descriptive reading of one public catalogue, coded by its own author — not usage telemetry, not a controlled experiment, and the author says so directly, including that “there is no public ROI study for a solo developer, loops are expensive, and usage patterns differ sharply between the token-rich and the token-poor.” Read the percentages as a portrait of what early adopters chose to publish, not as measurements of what works. That honesty is to the paper’s credit, and it leaves the field’s central quantitative questions — cost per loop-hour, failure rates, unattended-run safety — open.

The software factory: what’s substantiated and what’s narrated

The org-scale version of the story belongs to vendors, and the attribution should stay attached. At the World’s Fair, Warp CEO Zach Lloyd pitched the factory as automating “the main loop of software engineering: triage, specification, implementation, review, verification, shipping and monitoring,” with configuration as the control surface: “You choose your repositories, the parts of the software lifecycle you want to automate, and the points where humans should be brought into the loop.” Tereza Tížková of Factory — the startup whose brand is the metaphor itself — defined it as “the whole loop, the whole lifecycle of developing software with autonomy.” Lloyd’s summary of the engineer’s future role is that you end up “building the system that builds the product.”

What actually exists behind the framing, per the vendors’ own materials: Warp’s Oz (launched February 2026) is an orchestration platform — CLI, API, SDK, sandboxed cloud environments, a built-in scheduler — and Warp’s launch post reports agents writing 60% of the company’s own pull requests, plus an internal fraud bot whose PRs stopped nearly $60K of fraudulent usage. Cursor’s enterprise motion is a forward-deployed-engineering org in the Palantir mold — engineers embedded in customer systems across financial services, telecom, and semiconductors, with a stated plan to grow the team tenfold by December. Microsoft’s version, EVP Jay Parikh’s “agent factory”, is the same metaphor at hyperscaler size — complete with a Bill Gates anecdote that he’d envisioned a “software factory” since 1975.

Now the ledger. Every concrete number in the factory story here is vendor-internal dogfooding (Warp’s 60%, Warp’s $60K) or a hiring plan (Cursor’s tenfold). The customer-facing trajectory is explicitly aspirational — Lloyd frames it as a question, “Instead of merging 20% of pull requests automatically, can we get to 30%, 40%, 50% or 60%?” — and his flagship claim is a prediction, not an observation: “My prediction is that every significant software project will have some engine of code — something resembling a factory — continuously driving it forward.” No independent, customer-measured case study of a software factory operating end-to-end appears in any of this coverage; public evidence beyond vendor self-reports remains thin. And the friction report comes from the vendors themselves: Cursor’s VP of forward-deployed engineering says enterprise adoption stays concentrated among the 10–20% of enthusiastic early adopters unless top-down commitment and internal champions push it wider — which is a people problem, not a platform one. The counter-theme ran through the factory talks too: Lloyd concedes “there will still be a remaining percentage of work done by people because it is too difficult, ambiguous or dependent on greenfield thinking,” Sierra’s Natalie Meurer noted that “most customer-specific work takes place at the orchestration layer rather than in the models themselves,” and OpenAI’s Alexander Embiricos tied multi-agent value to connecting agents into review and deploy — the humans-steering caveat coming from the factory’s own proponents.

The metaphor’s next lap

“Software factory” has a long record of promising industrialization and delivering organization. Hitachi opened its Software Works in 1969; System Development Corporation adopted the model in 1975; NEC, Toshiba, and Fujitsu followed in 1976–77; and Microsoft’s 2004 Software Factories methodology (Greenfield and Short’s model-driven, product-line approach) was the term’s third mainstream run. The Department of Defense revived it in the late 2010s for organizations like Kessel Run — whose founding director, Enrique Oti, pushed back on the label himself: “Kessel Run is the term being used by the Department of Defense for a software factory. I don’t like to think of it as a software factory, but really, Kessel Run is an organization.”

That history is the right prior for the 2026 revival. Each earlier lap discovered that the factory’s hard parts were process, verification, and org design rather than the production machinery — and each settled into being a set of practices inside an organization rather than an automated line. The genuinely new ingredient this time is that the workers are software: loops really can run overnight, and the loop specification gives the factory floor a unit it never had. What hasn’t changed is where the difficulty lives. The corpus data and the vendors’ own caveats both locate it in verification design and human steering — the same places it lived in 1975. Treat the factory as a direction the tooling is pointed, not a thing you can currently buy whole.

Practical Implications

Adopt the artifact now; defer the factory. Writing loop specifications costs little, and the downside of piloting one is bounded: version them in the repo next to the skills they call, and review them like code. A minimum viable spec is six fields — trigger, goal, check, stopping rule with named terminal states, budget, and a state file — which is exactly the shape the Loop Library templates and the paper’s anatomy converge on. The org-scale platform decision can wait for non-vendor evidence.
Design the check before the instruction. The consistent lesson across the paper, Osmani, and Willison is that loop value concentrates in the verification step. Prefer deterministic, level-1/2 checks (exit codes, schemas, golden outputs) for anything that runs unattended; if the best available check is a model-as-judge rubric, keep the loop supervised and sample its judgments. Freeze the yardstick so turn N and turn N+10 are comparable.
Enforce terminal-state discipline. Give every loop named outcomes — success, no-op, blocked, stalled, exhausted — and make the harness report which one occurred. An exhausted budget that reports success is how overnight loops quietly pass incomplete work off as done; a no-op state is what lets a scheduled loop recognize there’s nothing to do and exit cleanly (true idempotency still has to be designed into the operations themselves). This is the cheapest upgrade over a bare while true, and it’s the specific thing Huntley’s original Ralph lacked.
Budget everything, and treat memory as curated state. Cap iterations, wall-clock, and spend per run. Keep loop memory in files the loop reads and writes, and prune it deliberately — the paper’s warning that ungoverned accumulated lessons can underperform no memory at all matches Huntley’s broken-codebase mornings.
Treat factory procurement as change management first. Cursor’s own deployment lead reports adoption pooling among the 10–20% of early adopters; an orchestration platform doesn’t fix that. Follow the trajectory Lloyd himself describes: start with low-risk repos and narrow task classes (dependency bumps, doc sweeps, triage), measure the auto-merge rate you can actually tolerate, and expand the automated share only as your verification ladder earns it.
Keep prompting for the ambiguous work. Loops fit goal-with-check work; they’re a poor fit for design, architecture, or anything where “done” resists a machine check — that work needs human-governed iteration, not unattended completion. Osmani’s “cognitive surrender” names the failure mode — letting the loop run so you can stop having an opinion. The loop specification is leverage when it encodes judgment, and an abdication when it substitutes for it.

Open Questions

Where’s the ROI line? The paper is blunt that no public cost study exists, and loop economics split sharply between subscription-flat and per-token users. At what loop-hour cost does an overnight documentation sweep stop paying for itself? Nobody has published the arithmetic.
Who reviews the loops? Once loop specifications carry approval authority (auto-merge on green), they become governance artifacts — but public practice on loop review, ownership, and audit is essentially undocumented. A repo full of loops has the same erosion risks as a repo full of unreviewed CI config, with an execution engine attached.
What’s the injection surface of event-triggered loops? The corpus says triggers are 78% manual today; the factory vision is event-driven. An event-triggered loop is an agent that starts itself on untrusted input, and the security analysis of that pattern hasn’t caught up to the enthusiasm.
Can stopping rules get principled? Stalled-versus-exhausted detection is currently heuristic (budget ceilings, stagnation counters). Whether plateau detection can be made reliable enough to trust for unattended multi-day loops is unresolved.
Does the factory consolidate or dissolve? CI didn’t stay a product category — it became a practice every platform absorbed. Whether “software factory” ends up a durable platform layer (Oz, Factory, agent HQs) or dissolves into ambient capability inside harnesses and forges is the open competitive question; the historical laps of the term argue for dissolution into practice.