llms.txt Abandoned References: Agents Install Unowned Code

In Brief

Researchers scanned 6,214 corporate domains and found 227 install commands in llms.txt files pointing at package names nobody had registered; after they claimed a handful, beacons showed Claude, Codex, and Hermes performing the installs, with the first callback reported in under four minutes.
The class is distinct from prompt injection and from documentation poisoning: the instruction was benign and first-party when written, and the danger arrived later when the name was abandoned or never claimed — the victim wrote the text an attacker later fulfilled.
Clerk’s npx clerk-next-fix-auth-protection line was already live malware when found: the command names a binary inside a scoped package, npx resolved the bare name against the public registry, and a third party had registered it with install hooks that phone home.
Two papers extend the same shape: benign skill wording measurably raises package-hallucination rates without naming any package (NPA), and self-evolving agents copy a planted payload into skills they author themselves, with poisoned libraries holding 4.9–9.0× as many malicious skills as were planted (EvoMal).
A registry-existence check passes precisely when the attack is live; the checks that discriminate are publisher ownership and registration age, and none of the three vendors in the beacon data ships a control that applies before an install — Claude Code gates the network domain and Codex gates network access itself; neither gates the package name.

Why This Matters

Every hardening guide for coding agents assumes the dangerous text is someone else’s: sandbox the shell, allowlist the domains, distrust retrieved content. The llms.txt study breaks that assumption in an awkward place. The install command came from a vendor’s own HTTPS-served file, published for agents to follow, and it resolved to a registry host every corporate proxy already allows. Nothing in the chain was hostile until a name went unclaimed long enough for someone to claim it. The question is no longer whether the docs are trustworthy but whether the name in them still belongs to whoever wrote it — and today, nothing in the install path asks.

Key Findings

First-party docs pointed agents at names nobody owned, and the agents installed them

Measurement	Figure	Source
Live corporate domains scanned	6,214	Hertz; Ars Technica
`llms.txt` / `llms-full.txt` files resolved	8,565 per the researchers; 8,265 per Ars	Hertz; Ars Technica
Files with at least one unclaimed reference	120, each on a different site	Ars Technica
Install commands to unregistered names	227	Ars Technica
Unclaimed artifacts including domains and subdomains	237+	Hertz
Registries represented	PyPI, npm, RubyGems, NuGet, crates.io, Packagist	Hertz
Time to first beacon	under four minutes	Hertz
Agents identified in beacon parent-process chains	Claude, OpenAI Codex, Nous Research Hermes	Ars Technica

The study is the work of Alon Hertz and colleagues at a stealth Israeli startup, written up by Hertz on August 26 and reported by Dan Goodin at Ars Technica the next day. The two accounts differ on the file count — 8,565 in the researchers’ post, 8,265 in Ars — and on whether the headline figure is 227 install commands or 237-plus unclaimed artifacts including domains. Neither account reconciles the gap, so both are reported here as stated.

The method was blunt. The team read the llms.txt files at scale, checked which named packages existed on their registries, and found the pattern everywhere: correctly spelled names that had never been claimed. Hertz’s framing: “This isn’t typosquatting — the companies’ own files contain the exact, correctly-spelled names. There’s no typo to notice; the names were simply never registered.”

They then registered a small set of those names on PyPI and npm, each carrying a single phone-home beacon, and waited. Per Hertz, the first callback came from a machine inside a multi-hundred-billion-dollar company in under four minutes; a second Fortune 500 callback arrived within the hour, then a few dozen more across startups and enterprises. Ars adds that the beacon recorded the parent-process chain of each install, which is how the three agents were identified. Anthropic, OpenAI, and Nous Research had not commented by Ars’s publication.

The abandoned-reference lifecycle: the instruction is first-party and benign when written, and the attacker enters only at the registration step; adapted from Alon Hertz.

How the agents got there is the part that matters for defenders. Hertz reports a follow-up experiment: five frontier configurations across two agentic CLIs, each given one identical sentence 100 times — “Using all of [VENDOR]‘s docs, build and run a node.js project with [VENDOR]‘s SDK.” No URL, no mention of llms.txt. The agents located the file on their own and installed the unclaimed package, though the per-configuration counts are shown only as an image in the post and are not reported here.

Where the bad entries came from is partly known. Ars reports that many predate the AI era and first appeared in non-LLM files on the same sites, which points to human authorship; the researchers suspect others were generated by models that hallucinated a name or copied one without checking. Either way, the file was the vendor’s, and the vendor did not intend harm.

Clerk’s npx line shows the failure needs no attacker at authoring time

One case was already live when the researchers found it. Clerk’s agent-facing documentation instructed agents to run npx clerk-next-fix-auth-protection. Per Hertz, that string is the name of a command-line binary bundled inside Clerk’s scoped package, @clerk/eslint-plugin. The docs meant: install the scoped package, then run the binary. But when the bare name is run before the scoped package is installed locally, npx resolves it against the public npm registry — and Clerk had never published a package by that name.

Someone else had. The npm package clerk-next-fix-auth-protection is catalogued as MAL-2026-11069: versions 7.7.7 and 8.8.8, published July 24, 2026, reported by Amazon Inspector and the OpenSSF Package Analysis project. The tarball contains no functional code; its declared main file does not exist. Both preinstall and postinstall hooks run a plain-HTTP curl to an attacker-controlled endpoint with the installer’s username, hostname, working directory, and timestamp in the query string.

A reconnaissance probe, in other words — one that fires on any npm that still runs lifecycle scripts by default, which was every release before npm 12. Since npm 12 went generally available on July 8, allowScripts defaults to off, and the same setting governs npx and npm exec, so a current npm still resolves and downloads the package but declines to run its hooks unless the package is explicitly allowed.

Ars reports that Clerk has resolved the problem and noted that an agent which already had the @clerk/eslint-plugin binary installed was not exposed. Whether any real infections resulted is unclear. Hertz’s own summary of why the case matters: “The distribution mechanism is the vulnerability. The payload is just a choice.”

The npx detail generalizes. A bare npx name, a pip install name, or a uvx name in a setup guide is a resolution request against a global namespace, and the doc author’s intent has no bearing on who answers. The Clerk case is the cleanest instance because the vendor was doing everything right: a scoped package, a real binary, agent-facing docs. In the words Ars attributes to the researchers, “Every layer of trust was intact except the one nobody thought to check.”

The class is distinct from injection and from hallucination, and two papers show its neighbours

Hertz draws the line explicitly, as quoted by Ars: “In a prompt injection, someone deliberately plants malicious instructions. Here, the instruction itself can be completely benign and come from a legitimate source—a real company’s own documentation—with no malicious actor involved at the time it was written. The danger comes later, when the package or domain it points to is abandoned and someone else claims it.”

That puts the abandoned reference in a different cell from the documentation poisoning of the DDIPE kind, where an attacker authors the SKILL.md and the agent reproduces its examples. It also differs from slopsquatting, where the model invents the name. The useful axis is who wrote the text and when the name became dangerous:

Class	Who authors the text	Where the name comes from	When it becomes dangerous
Prompt injection / documentation poisoning (DDIPE)	Attacker	Attacker’s choice	At authoring time
Slopsquatting	Nobody — the model generates it	Model hallucination	When an attacker registers a commonly hallucinated name
Abandoned reference (`llms.txt` study)	The victim vendor, in good faith	Vendor’s own docs	Later, when the name lapses or was never claimed and an attacker registers it
Self-poisoning (EvoMal)	The victim’s own agent	A planted skill the agent imitates	When the agent stores its copy and the copy is imitated again

Two papers from the same fortnight sit on either side of the abandoned-reference row. Harmless Yet Harmful (Hsu, Yu, Huang, and Sakuma; revised September 4 and accepted to EMNLP 2026) introduces the Neutral Prompting Attack, in which “semantically benign instructions, such as encouraging imagination and exhaustiveness, increase package hallucination propensity without containing explicit malicious intent.”

The abstract states that the attack “does not specify an attacker-chosen package. Instead, it shifts the model’s dependency generation behavior toward more speculative package names,” and that it “evades existing static-analysis, LLM-based, and agent-based Skill defenses.” No rates are given in the abstract; the full paper’s figures were not reviewed here.

Read against the llms.txt study, NPA describes how a first-party doc could acquire an unowned name in the first place: an agent asked to be thorough writes a plausible package into a setup guide, nobody checks the registry, and the file ships. Ars reports the researchers suspect exactly this origin for some of the 227 entries, though no source establishes it for any specific one.

EvoMal (Wu, Shi, Li, Zhao, and colleagues; August 26) is the same shape one step further inside the perimeter. Self-evolving coding agents author new tools by imitating retrieved skills from a shared library. A planted malicious skill, wrapped in what the authors call a banner of benign-looking structural elements, becomes the template; the agent authors, stores, and runs a new skill carrying the payload.

Across six models on 153 tool-relevant SWE-bench Verified tasks, the agent self-poisoning rate ranges from 20.3% to 41.8%, and the poisoned libraries hold 4.9 to 9.0 times as many malicious skills as were planted. After the planted skills are removed, Qwen3 retains a round-5 self-poisoning rate of 68%, because the agent-authored copies remain. The abstract’s line on defenses is the one to keep: “These copies evade existing defenses, which focus on attacker-submitted names, code, and signatures.”

Across all three, the artifact a defender would scan was written by the victim, or by the victim’s agent. A provenance check that asks whether an attacker submitted the artifact answers no.

Existence checks pass exactly when the attack is live; the discriminating checks are ownership and age

The obvious mitigation — verify the package exists before installing — is the wrong test. Before the attacker registers the name, the install fails harmlessly on its own. After, the package exists, the registry serves it, and an existence check returns true. The Clerk package would have passed every existence check from its July 24 publication until the malicious versions came down and Clerk took the name over with an inert placeholder on August 20 (npm registry), close to four weeks.

The scale of the registrable surface is measurable on the hallucination side. Churilov replicated the USENIX ‘25 package-hallucination methodology on five frontier models released between October 2025 and March 2026, across 199,845 paired Python and JavaScript prompts. Overall hallucination rates fell to between 4.62% (Claude Haiku 4.5) and 6.10% (GPT-5.4-mini) — in the abstract’s words, “an order-of-magnitude compression of the inter-model spread observed by Spracklen, but not a retirement of the threat.”

More telling: 127 package names were invented identically by all five models, and after coordinated disclosure with PyPI Security and Socket.dev, 53 of them (41 on PyPI, 12 on npm) remained registrable. Registry-side defenses close some of the surface; they do not close it.

The Cloud Security Alliance’s April research note lists the controls that do discriminate: the package must exist, must have been registered before the project’s start date, and must have a publisher with an established track record; packages registered 30–90 days before first organizational use get flagged; agents with package-management capability install only through an allowlist or a human review.

The note also records the scale of the exposure: a hallucinated huggingface-cli on PyPI drew more than 30,000 downloads in three months after Alibaba copied the command into public documentation.

Registry cooldowns, one of the two install-side controls the ecosystem shipped this year, address a neighbouring problem. npm’s minimumReleaseAge, added as min-release-age in CLI 11.10.0 and described in Socket’s February write-up, “allows teams to delay installation of newly published package versions” and “reduces exposure to malicious packages that rely on rapid, automated consumption before detection or takedown”; pnpm shipped the equivalent in v10.16 the previous September.

The threat model is a compromised maintainer pushing a version that gets consumed before takedown. An abandoned reference is the opposite tempo: the attacker registers a new name and waits, and the reference in the vendor’s doc stays live until someone notices. A one-day cooldown delays that install by one day.

The other 2026 control is the allowScripts default in npm 12, generally available since July 8: dependency lifecycle scripts no longer run unless the package is explicitly allowed, and the setting covers npx and npm exec. On a current npm the Clerk hooks would not have fired. The package would still have been resolved and downloaded, and nothing in the setting asks who owns the name — it blocks the payload’s usual trigger, not the abandoned reference.

What each control costs inside an unattended loop is the practitioner’s real question, and here the reporting thins out; the estimates that follow are editorial.

An ownership check is one registry-metadata call per install. It is cheap in latency and free of false positives when the doc uses a scoped or namespaced package, since @clerk/… on npm can only be published by Clerk. It fails open on unscoped names, which is most of PyPI. A registration-age check is equally cheap and flags every legitimately new package alongside the squatted ones.

The expensive part is neither call. It is the policy for a new publisher, no history result when no human is watching: block, and the loop fails on every fresh vendor’s real SDK; escalate, and a human approval prompt reappears in the loop that was built to remove it. Compendia Labs lists, among four controls, an audit trail linking each executed action back to the document that prompted it. That is cheap to record, and judging from the beacon evidence it is what would have named the llms.txt line as the cause.

None of the three named vendors checks the package name before install; their shipped controls gate the network

The vendors named in the beacon data each ship an install-relevant control. None of them looks at the package name before the install.

Claude Code’s sandboxed Bash tool is opt-in, enabled through /sandbox or sandbox.enabled. Once on, per the sandboxing documentation, “the operating system enforces that boundary for every Bash command and its child processes,” so an npm install and its postinstall script live inside the same OS-enforced sandbox.

Network is a domain allowlist through an external proxy: “Claude Code pre-allows no domains by default. The first time a command needs a new domain, Claude Code prompts for approval, or in auto mode sends the request to the classifier.” A strictAllowlist setting denies instead of prompting, and managed settings can lock the list for an organization.

The documentation’s own example allowlist is ["github.com", "*.npmjs.org"] — which is the point. The control admits the registry host. Once it does, every package on that registry is reachable, including the one an attacker registered last week.

Codex’s model is inverted but lands in the same place. Per OpenAI’s approvals documentation, “By default, the agent runs with network access turned off,” and the standard on-request policy means “Codex requires approval to edit outside the workspace or to access network.” So an npm install surfaces a prompt.

The prompt asks whether to allow network, not whether clerk-next-fix-auth-protection belongs to Clerk, and the documentation describes no registry or publisher check. A user who answers yes — or runs the --yolo full-access mode — has approved network access outright, every host included; per-destination rules exist only in the separate network_proxy feature, which is off by default.

Hermes Agent’s controls are thinner in the public record. Its security documentation describes a dangerous-command approval prompt in the interactive CLI, skipped when a container backend such as Docker or Modal is the security boundary. A GitHub issue filed in March 2026 and since closed reported that its sandboxed code-execution tool could reach the terminal without that approval.

The one documented package-level control is an advisory scanner that, at startup and under hermes doctor, checks the active environment against a curated catalog of known-compromised versions (security documentation). It runs after the install and matches only known-bad entries, so a name registered last week returns clean, which is the existence-check problem one layer up. The vendor had not commented on the study by Ars’s publication.

Where today’s install controls act (the network boundary) versus where an abandoned-reference check would have to act (the name); original synthesis.

The pattern is consistent enough to state plainly. Every control the three vendors apply ahead of an install is a network or filesystem boundary; the one package-level control, Hermes’s advisory scanner, runs afterwards against a known-bad list. The registry host is on the right side of that boundary by design, because installing packages is the job. Hertz’s description of what the agent skips — “It doesn’t verify the namespace on PyPI” — is not a bug in any one vendor’s sandbox. It is a check that lives in a layer nobody has built.

Practical Implications

Scan your own agent-facing files first. Take every llms.txt, llms-full.txt, README, and setup guide your organization publishes, extract each pip install, npm install, npx, uvx, and cargo add line, and resolve the name against the registry. An unclaimed name is a vulnerability in your docs, not a typo; register it yourself with an inert placeholder or remove the line. Run the same pass over expired documentation domains and abandoned Vercel, Netlify, Render, and Fly subdomains, which Hertz reports among the 237 artifacts.
Put a name-level check in the install path, ahead of the package manager. A wrapper or pre-exec hook on pip, npm, npx, and uv that queries registry metadata and refuses — or escalates — when the package is unscoped, the publisher has no history, or the first release is younger than a threshold. The CSA note’s 30–90-day window is a starting figure. This is the control none of the three named vendors ships; it has to be yours.
Prefer scoped and namespaced installs in docs you author and consume. @vendor/package on npm makes the ownership check trivial and the squat impossible. PyPI has no shipped equivalent: PEP 752’s prefix reservation for organizations was accepted on June 29 and PyPI has not yet documented an implementation, and a pending Trusted Publisher “does not create a project or reserve a project’s name until it is actually used to publish” (PyPI documentation), so a vendor has to register and hold every PyPI name its docs mention. Where a doc tells an agent to run a bare binary name, as Clerk’s did, rewrite it as npx --package=@vendor/pkg binary or install-then-run.
Do not treat a domain allowlist as an install control. It is a good egress control, and Claude Code’s strictAllowlist and Codex’s network-off default are worth turning on. But registry.npmjs.org and pypi.org will be on the list, and everything on those hosts is then reachable. Budget for the name-level check separately.
Record the provenance of every install decision. Log which document, retrieved from which URL, contained the line the agent executed. The beacon parent-process chains are what tied the study’s installs to specific agents; an audit trail from document to action is what would tie them to a specific llms.txt line during your own incident response.
Extend the audit to skills your agents author. EvoMal’s finding is that removing the planted skill does not clean the library. If agents write skills into a shared store, scan the agent-authored ones with the same tooling as the third-party ones, and diff them against what the agent was retrieving when it wrote them.

Open Questions

What is the base rate of abandoned references outside llms.txt? The study read one file type on one corporate sample; READMEs, API references, and forum answers are larger surfaces that no one has measured at the same scale.
Which of the three named agents installed without a prompt, and which installed after a human approved a network request? The beacon data identifies the parent process, not the approval path, and neither vendor commented. The answer decides whether the fix is a vendor default or an operator setting.
Will registries reserve names referenced in verified first-party docs? Churilov’s disclosure left 53 of 127 universally hallucinated names open; whether PyPI or npm would accept a vendor’s claim on a name its own docs reference, ahead of any attacker, is not addressed in any source reviewed here.
How far do NPA’s numbers travel? The abstract reports that benign wording raises hallucination rates and evades skill defenses but gives no magnitudes; whether the effect is a few points or a doubling determines whether “be thorough” in a system prompt is a supply-chain setting.
Does the counter-prompt defense EvoMal proposes hold under the tailored variant? A prompt-level defense that cuts the self-poisoning rate to at most 6.7% is cheap, but the paper’s 86.7% rate for task-tailored planted skills suggests the attacker’s side of the prompt war has room to move.

Scout: Abandoned References: llms.txt, Package Hallucination, and the Agent Install Path