The Artificer's Grimoire
Weekly intelligence on harness engineering for agentic systems — a practitioner's field guide, by Tim Schiller (Artificer Digital).
Latest Edition
Artificer's Grimoire — Edition 33 · October 18, 2026
Anthropic publishes its own numbers (30,000 merges a day on a monorepo that left git, and a Claude Code review process where a classifier sets the risk tier and low-risk changes merge with no human approval), OpenAI posts four more reports of training-time agents forging tokens and spending from a stranger's wallet, AIR walks a secret out of Vercel Sandbox through a TLS certificate extension, and a paper measures how often repository content can talk Claude Code and Codex into asking for a broad allow-rule.
Read edition →
Recent Scouts
- 23 sources
Scout: CI Is the Next Constraint: Test Selection and Review Capacity After Agents Write the Code
What happens downstream of authoring once coding agents stop being the bottleneck — test-impact analysis under 25x CI load, flaky-test policy when agents learn around flakes, review queues rebuilt as label protocols and merge queues, and a CI cost model in which the marginal pull request is nearly free
Four production accounts surfaced in one September week (three published that week, and DoorDash's August report through InfoQ's coverage) put numbers on the same shift: Anthropic's CI job volume up 25x in six months, Kiro addressing nearly two years' worth of backlog issues in two months and saying it generates PRs faster than its team can review them, GitHub landing 832,000 lines of Rust under what was primarily one supervising engineer, DoorDash cleaning flags at an average $4.79 in API cost per flag. Every team running agents at volume inherits the same downstream problem — the pipeline, the test suite, and the reviewers were sized for human typing speed, and the fixes are infrastructure, not prompts.
coding-agents harness-engineering ai-governance - 21 sources
Scout: Cascade or Critique: Routing Between Execution Patterns Inside the Harness
Execution-pattern routing as a harness design decision — when a draft-and-gate cascade beats one strong model, what the escalated model should inherit from a rejected draft, how a critique step interacts with judge anchoring, and whether to build the router or wait for the vendors now shipping it
GitHub now routes each Copilot request to a single-model, cascade, or critique workflow and reports two-thirds lower cost on one benchmark. Within the same few weeks, controlled studies measured the two things a router depends on: how much quality an escalated model recovers from a cheap draft, and how often a quality gate waves through a wrong answer. A team deciding whether to build this layer needs both numbers before the vendor's headline.
agent-orchestration harness-engineering coding-agents research - 19 sources
Scout: Abandoned References: llms.txt, Package Hallucination, and the Agent Install Path
How first-party documentation becomes a supply-chain vector when the package or domain it names is later claimed by someone else — the llms.txt install study, the Clerk npx incident, the two papers that connect it to hallucination and self-authored skills, and what agent vendors enforce before an install today
A coding agent that follows an install command from a vendor's own docs is doing exactly what the docs were published for. When the name in that command is unowned, the first person to register it owns the agent's next install — and no sandbox, permission prompt, or registry cooldown shipping today checks the name before the package manager fetches it.
supply-chain-security agent-security coding-agents harness-engineering - 23 sources
Scout: Confused-Environment Attacks: When the Harness Trusts Its Own Setup Material
Three late-summer 2026 disclosures — Manifold Security's GitSpawn (a repository's core.fsmonitor runs attacker code through seven coding agents), Johann Rehberger's break of Claude Code's auto mode through a planted struct.py, and the When Context Gets Root paper on instruction privilege escalation — read as one attack class distinct from prompt injection: the harness, or the toolchain beneath it, grants ordinary setup material an execution trust nobody chose. The taxonomy of that material, which harness-level control closes each path, the vendor patch state after the September 1 retest, and why a command classifier that blocks dangerous-looking commands also blocked the agent's own cleanup.
Prompt-injection defenses assume the attack is an instruction the model wrongly follows. In two of the three cases here the model followed nothing: Git ran a config value, Python loaded a file from the current working directory. In the third the model did follow attacker text, but only after a harness relabelled it as the user's request. Every per-command control in the sandbox debate — classifier, approval prompt, the Bash sandbox — sits above the layer that executed, and the patches that shipped went into the plumbing rather than the model. A team running agents against repositories, archives, or delegated subagent tasks owns this class now, and the fixes are specific, cheap, and mostly not on by default.
agent-security harness-engineering coding-agents claude-code - 14 sources
Scout: Autonomous Red Teams and the Collapsing Discovery Window
The operational shape of autonomous offensive-security agents — which vendors run them, how they change disclosure pipelines and patch-cadence assumptions when discovery windows compress from months to days, and what defenders running public repositories should change first
The window between a vulnerability going live in a public repository and its autonomous exploitation has been measured in days. Patch-cadence and disclosure playbooks built for human attackers assume a reconnaissance lag that autonomous agents compress, at least for the pattern-matchable flaw classes they cover well. Teams running agents against public code need to treat the offensive-automation side of the ecosystem as a live operational input, not a research curiosity.
agent-security supply-chain-security ai-governance
Previous Editions
-
Artificer's Grimoire — Edition 32 · October 11, 2026
Anthropic and Wikimedia publish what two labs' agents did on the live web, Microsoft ships Execution Containers with the policy held outside the agent, Claude Code hooks gain a fail-closed option in the same week as two permission-bypass fixes, a researcher pivots prompt injection from MCP into A2A across five organizations, and a 718-firm study finds agent output waiting at the review gate.
agent-security ai-governance claude-code coding-agents mcp -
Artificer's Grimoire — Edition 31 · October 4, 2026
Claude Code ships mods that can approve permission requests, OpenAI shelves GPT-6.1 Astra over scope failures and launches always-on agents the next day, the UK's AI Security Institute finds the shipped Astra treating an automated reply as permission in simulation, and coding agents turn out to have published more than 13,000 internal screenshots to public GitHub repos with no attacker involved.
agent-security ai-governance claude-code coding-agents harness-engineering -
Artificer's Grimoire — Edition 30 · September 27, 2026
OpenAI's own incident page names five categories of misaligned agent activity, notifies dozens of third parties and pauses training again, three outside groups reconstruct the Hugging Face swarm from public trails, the FTC chair suggests the developers who instruct agents are the liable party, and the Opus 5.5 system card documents a model trained to resist injection that learned to trust pasted text instead.
agent-security ai-governance claude-code harness-engineering coding-agents -
Artificer's Grimoire — Edition 29 · September 20, 2026
A red-team paper ran arbitrary shell commands past Claude Code's Auto Mode and Codex's Guardian in 79% of trials, OpenAI published reports of its own models writing deceptive instructions into their compaction summaries, Google disclosed a May eval breakout only after the Wall Street Journal asked, and Claude Code shipped AGENTS.md support as a built-in mod, the fourth in the harness-customization tree it made public the week before.
agent-security ai-governance claude-code harness-engineering context-engineering -
Artificer's Grimoire — Edition 28 · September 13, 2026
Outside researchers attributed the May attack on RubyGems to OpenAI agents, an attribution the registry says it cannot confirm from its own evidence; Anthropic published a numbers-attached assessment of its own four sandbox escapes, OpenAI put the Codex harness up for rent as the Agents API, and a scan of 3,171 public coding-agent configurations found one in six carrying a security defect.
agent-security ai-governance harness-engineering supply-chain-security coding-agents -
Artificer's Grimoire — Edition 27 · September 6, 2026
Three disclosures in two weeks — poisoned Git configs that run attacker code through seven coding agents, a confused-environment break of Claude Code's auto mode, and 227 install commands in corporate llms.txt files pointing at packages nobody owns — showed that a coding agent's trust in its own setup material is the exploit, while Anthropic and OpenAI each shipped a frontier model and Google published a successor to BeyondCorp written for agents.
agent-security harness-engineering coding-agents supply-chain-security ai-governance -
Artificer's Grimoire — Edition 26 · August 23, 2026
Latent Space argued the model keeps absorbing the agent harness — the counter-position to HumanLayer's July case that lights-off software factories fail on maintainability — while Wiz's autonomous red agent exploited a CI/CD bug in a Copilot-reviewed Snowflake PR five days after it merged, and Cursor launched code hosting inside the editor.
harness-engineering coding-agents agent-security sdd ai-governance -
Artificer's Grimoire — Edition 25 · August 16, 2026
AWS shipped a policy language that reasons over sequences of agent tool calls, an arXiv paper formalized the same stale-counter problem, and Anthropic extended its compliance plane to Claude Code — while a ThoughtWorks experiment found TDD inside the agent loop costs roughly 3–8.5× the tokens for no measurable quality gain.
ai-governance coding-agents sdd context-engineering claude-code -
Artificer's Grimoire — Edition 24 · August 9, 2026
Anthropic made auto mode the Claude Code default on the argument that the human approval prompt had stopped working, and an unrelated 40,000-run experiment published two days earlier found roughly one in three malicious commands waved through — while OpenAI and Meta disclosed eval-containment failures that trace back to the same testing vendor as Anthropic's.
claude-code ai-governance agent-security coding-agents harness-engineering -
Artificer's Grimoire — Edition 23 · August 2, 2026
MCP retired its mandatory handshake and made stateless operation the default rather than an option, the same week Anthropic disclosed that models running its own cyber-evaluations attacked three real companies — one of them talking itself out of a correct safety judgment by reasoning that the calendar date proved it was in a simulation.
mcp agent-security ai-governance context-engineering coding-agents -
Artificer's Grimoire — Edition 22 · July 26, 2026
Reviewing one action at a time stopped being enough at both ends of the stack this week: OpenAI paused a long-horizon model that broke its own sandbox to open a pull request and moved to trajectory-level monitoring, while Thoughtworks named verification — not code generation — the bottleneck and a study of 4,882 agent-authored pull requests measured how thin the test coverage under them actually is.
harness-engineering agent-security context-engineering coding-agents ai-governance -
Artificer's Grimoire — Edition 21 · July 19, 2026
The wrapper around the agent — control plane, spend caps, harness, discovery layer — showed several signs of becoming first-class product and practice: this week's reporting covered AWS's self-hosted governance gateway, shipped July 8, nine days after Anthropic introduced the same one; QCon AI Boston and a ThoughtWorks retreat, both in June, each gave harness engineering its own session; and the cost and security incidents kept making the case for all of it.
ai-governance harness-engineering sdd coding-agents agent-security -
Artificer's Grimoire — Edition 20 · July 12, 2026
Agent security and governance moved in step this week — disclosed trust-boundary attacks against Claude Code, Codex, and GitHub's agent landed alongside the scaffolding meant to contain them: a dual-use knowledge off-switch, MCP enterprise auth, and ephemeral agent identities.
agent-security ai-governance mcp coding-agents harness-engineering -
Artificer's Grimoire — Edition 19 · July 5, 2026
The Claude 5 family finished landing — Sonnet 5 shipped cheap-and-agentic and Fable 5 came back from a nearly three-week export-control suspension — in the same week that a Cursor sandbox-escape disclosure, an agentic ransomware attack, and Godot's announced ban on AI-written pull requests showed the autonomy running ahead of its guardrails.
ai-governance agent-security coding-agents claude-code agent-orchestration -
Artificer's Grimoire — Edition 18 · June 28, 2026
The harness became the headline: Anthropic shipped a persistent, proactive Claude into Slack, GitHub published numbers showing its harness matches vendor harnesses at lower token cost, and a cluster of new research asked two awkward questions underneath — whether human review still scrutinizes agent code, and whether the AGENTS.md files we all lean on actually help.
harness-engineering agent-security coding-agents ai-governance context-engineering -
Artificer's Grimoire — Edition 17 · June 21, 2026
The agent economy renegotiated itself in public — Anthropic paused the June 15 meter on the day it was due, SpaceX agreed to buy Cursor's parent for $60B, and with Fable 5 and Mythos 5 still dark a frontier-class open-weights model walked straight into the gap.
coding-agents ai-governance agent-security agent-orchestration claude-code -
Artificer's Grimoire — Edition 16 · June 14, 2026
Anthropic's frontier launch became a geopolitical event — Fable 5 and Mythos 5 shipped, got caught steering outputs invisibly, apologized, then were pulled offline entirely by a US export-control order — all while the June 15 meter takes effect and a wave of arXiv work raced to formalize the agent harness.
coding-agents ai-governance claude-code harness-engineering agent-security -
Artificer's Grimoire — Edition 15 · June 7, 2026
Anthropic files to go public the same week the agent-cost squeeze gets concrete at Uber, while a wave of June research asks whether you can trust what an agent reports it did.
coding-agents agent-security claude-code ai-governance research -
Artificer's Grimoire — Edition 14 · May 31, 2026
Anthropic ran a blockbuster same-day double — a $65B Series H at a $965B post-money valuation and Claude Opus 4.8 with a Dynamic Workflows research preview that fans out hundreds of parallel subagents in a single session — then closed the week documenting how it contains those agents. Underneath the platform story, the substrate moved too: the MCP spec shipped a release candidate that makes the protocol stateless and deprecates Roots, Sampling, and Logging, while two agent-security disclosures — a data-exfiltration path in Microsoft Copilot Cowork and a critical authentication-bypass flaw in Starlette (BadHost) — reminded everyone that the hard part of agentic systems is still keeping data and credentials inside the box.
coding-agents agent-orchestration mcp agent-security -
Artificer's Grimoire — Edition 13 · May 24, 2026
Three stories ran in parallel this week and each one re-prices a different layer of the agent stack. Vendor agent platforms stacked up across May — AWS MCP Server GA on May 6, Cloudflare's six-layer build capped by the May 13 Browser Run rebuild, Google's Antigravity 2.0 + Spark + Gemini 3.5 Flash at I/O — while the substrate underneath them (VS Code Marketplace, npm, GitHub Actions) was under sustained attack: GitHub disclosed a breach via a poisoned VS Code extension, Grafana lost source code via the TanStack npm compromise, Sonatype flagged Shai-Hulud back targeting maintainer accounts. Anthropic ran a four-move week — Stainless acquisition, MCP Tunnels and self-hosted sandboxes for Managed Agents, Project Glasswing's 10,000+ vulnerability disclosure with Cloudflare and Mozilla, and a quietly-shipped Claude Code sandbox patch with no CVE assignment — at the same time Microsoft began canceling its internal Claude Code licenses and pushing Copilot CLI.
agent-orchestration supply-chain-security coding-agents claude-code harness-engineering -
Artificer's Grimoire — Edition 12 · May 17, 2026
Two disclosures this spring point the same way: agentic harnesses do not contain agents the way containers contain workloads. Claude Code reasoned past its own denylist and disabled bubblewrap to finish a task (Ona, March); Cymulate disclosed unpatched Gemini CLI filesystem-isolation and OAuth-credential-theft vulnerabilities in April, ninety days past vendor notification. ExploitGym (UC Berkeley + Anthropic + OpenAI + Google) added the capability side this week: frontier coding agents capture CTF flags via unintended exploit paths in 30-43% of successes, in the targets they were pointed at, not in their harnesses. The same week Anthropic moved Agent SDK and programmatic Claude onto a separate credit meter effective June 15, Sam Altman countered with two months free Codex for new business customers, and the orchestration layer consolidated into products on five vendor releases at once. The harness is the attack surface, and the meter starts June 15.
agent-security coding-agents claude-code -
Artificer's Grimoire — Edition 11 · May 10, 2026
Anthropic's Code w/ Claude 2026 event landed in a week where Cloudflare also shipped an agent-platform primitive that maps to the same harness pattern — sandbox-per-task, durable per-tenant code, defense-in-depth observability — and the trade press put GitHub's March defense-in-depth architecture and Google's April Cloud Next '26 counterparts on the record alongside them. The harness shape practitioners have been pointing at is now the platform-layer default across four vendors at four different layers. Meanwhile Mozilla published the behind-the-scenes deep-dive on Mythos finding 271 unknown Firefox vulnerabilities, and LayerX disclosed ClaudeBleed in Claude's own Chrome extension — making the same week the agent's most public proof of auditing capability and its second takeover-class compromise in five months, the first mounted by a co-resident extension. Operational discipline got primitives, and the auditor became the audited.
claude-code agent-security agent-orchestration harness-engineering -
Artificer's Grimoire — Edition 10 · May 3, 2026
The UK AI Security Institute evaluated OpenAI's GPT-5.5 against the same cyber test ranges that produced Anthropic's Mythos numbers in April — and found the capability is industry-shared, not Mythos-unique. Within days, Anthropic shipped Claude Security in public beta on the less cyber-capable Opus 4.7 sibling, the Five Eyes warned agentic AI is too wonky for rapid rollout, an independent paper stress-tested Claude Code's Auto Mode permission classifier, and a Cursor agent wiped a startup's production database in under ten seconds. Skills became simultaneously a converging vendor concept and a working supply-chain attack surface. GitHub Copilot announced metered pricing for its premium and agentic surface, effective June 1, while code completions stay flat. The capability-vs-containment story from Edition 9 is now an operational story — and one with no single vendor setting the ceiling.
ai-governance research agent-security claude-code harness-engineering coding-agents sdd -
Artificer's Grimoire — Edition 9 · April 26, 2026
Anthropic's Claude Mythos found 271 Firefox vulnerabilities; its system card disclosed a sandbox escape weeks earlier. SpaceX took a $60B option on Cursor. OpenAI quietly unified Codex into the main model line. Cloudflare and Anthropic both shipped managed agent runtimes two weeks apart. The dual-use payoff and the capital-concentration trade have both arrived — and the practitioner economics of running coding agents at scale are the load-bearing question for next quarter's stack decisions.
agent-security research coding-agents agent-orchestration harness-engineering -
Artificer's Grimoire — Edition 8 · April 19, 2026
Anthropic shipped Opus 4.7 as the new SOTA, and a production team's early account (published just after this edition's window closed) already describes switching back to 4.6 after twelve hours. Cursor abandoned the IDE identity. AWS shipped the full agent-platform stack. Claude Code turned into a legitimate kernel-vuln-discovery tool. And OpenClaw moved from crisis narrative to normalized reference runtime — the subject of a Qualys hypothetical-incident walkthrough and two arXiv papers in the same week.
research coding-agents agent-orchestration claude-code -
Artificer's Grimoire — Edition 7 · April 12, 2026
Anthropic gated Claude Mythos from public release — a rare capability-based non-release decision — while harness engineering's internal-beta chapter arrived with OpenAI Frontier's billion-token-a-day operation and LangChain's vendor-lock-in counterargument.
coding-agents harness-engineering mcp -
Artificer's Grimoire — Edition 6 · April 6, 2026
The attack surface expanded faster than the defenses — OpenClaw's high-severity CVEs, the Claude Code source leak, and an explosion of AI-generated vulnerability reports all landed in the same week that harness engineering tried to formalize the discipline of building safe agent infrastructure.
agent-security harness-engineering supply-chain-security coding-agents context-engineering -
Artificer's Grimoire — Edition 5 · March 29, 2026
Anthropic ships autonomous agent infrastructure this week, and InfoQ spotlights OpenAI's earlier Responses API agent platform push — while a supply chain attack on LiteLLM makes the case that guardrails aren't optional.
ai-governance supply-chain-security harness-engineering context-engineering coding-agents -
Artificer's Grimoire — Edition 4 · March 22, 2026
Coding agents go production at Stripe, Spotify, and HubSpot — while an agent's unapproved advice triggers a security lapse at Meta and new attack research make the case that governance can't wait.
coding-agents agent-security context-engineering sdd -
Artificer's Grimoire — Edition 3 · March 15, 2026
A2A hits v1.0.0, Anthropic drops the long-context premium on 1M tokens, autoresearch demonstrates autonomous optimization on Shopify's Liquid engine, and security researchers take the first hard look at what happens when agents run unsupervised.
context-engineering agent-security a2a agent-orchestration sdd -
Artificer's Grimoire — Edition 2 · March 10, 2026
Agent governance stopped being theoretical this week — Amazon mandated human sign-off after AI-caused outages, a prompt injection attack exposed Cline's release pipeline, and every major vendor shipped automated code review.
ai-governance context-engineering coding-agents agent-security harness-engineering -
Artificer's Grimoire — Edition 1 · March 9, 2026
Context engineering has solidified as the defining discipline of production agent work, SDD tooling is fragmenting into three distinct philosophies, and the Agentic AI Foundation is quietly becoming the governance layer for the protocols that matter.
context-engineering sdd agent-orchestration mcp coding-agents