groundy
agents & frameworks

Can a Malicious AGENTS.md File Compromise Your Coding Agent? A Threat Model

AGENTS.md and .cursorrules are a write-once, compromise-many attack surface. arXiv 2607.15143 shows a deterministic pre-install check, not sharper models, stops it.

11 min···5 sources ↓

Yes, if you let it. arXiv 2607.15143 shows that the files coding agents read at setup time, from READMEs and requirements files to Makefiles and, by extension, AGENTS.md and .cursorrules, are a write-once, compromise-many surface. An attacker who controls repo-level setup text can redirect an agent to malicious packages or known-vulnerable versions, and the control that actually stops this is a deterministic pre-install check, not sharper model judgment.

How do setup instructions become an attack surface?

The paper’s central finding is that AI coding agents treat setup documentation as a trusted input and then act on it, installing packages and editing configuration without independent verification of names, sources, or known vulnerabilities. The attack surface is the gap between “the agent reads this file” and “the agent executes what the file says.” For setup files, that gap is effectively zero.

The edits an attacker needs are trivial. Changing one README line, one entry in a requirements.txt, or one Makefile target is enough to redirect an agent to an untrusted package registry or pin a known-vulnerable version. The file is read on every clone, so a single edit compromises every agent and every developer whose agent touches that repo. This is the write-once, compromise-many property the paper names, and it is materially worse than a conventional malicious dependency, which is evaluated once at install time and then sits in a lockfile. A poisoned setup file keeps re-arming on every fresh checkout.

The empirical work is grounded. The authors tested frontier models across twelve scenarios in five attack classes, each grounded in documented incidents, covering the npm and Cargo ecosystems. The headline result: nearly every tested model installs untrusted dependencies in source-based attacks. Blatant typosquats get caught reliably, but separator-confusion names like azurecore for azure-core slip through at rates that vary by harness and model. The asymmetry between the two failure modes is the part worth sitting with: the attacks that look obviously wrong are handled; the attacks that look plausibly right are not.

Why does the same model behave differently across harnesses?

Install-time safety is a property of the harness-model pairing, not the model alone. The same model that catches an attack through one harness installs it through another.

This is the finding that breaks the usual way teams reason about agent safety, which is to ask whether a particular model is robust to prompt injection. The paper’s data says that question is under-specified for setup-time attacks. What the model does with a suspicious package name depends on how the harness scaffolds the install step, what context it injects, what guardrails wrap the package manager, and whether a human-in-the-loop confirmation sits in front of the install. Change the harness and you change the outcome, even holding the model constant.

The mechanism is not mysterious. A harness that renders the proposed pip install command in plain text and waits for a keystroke gives the model, and the operator, a chance to notice. A harness that pipes the command straight into a shell because the README told it to removes that chance. The same underlying weights produce different behavior because the scaffolding around them is different. This is why a model that scores well on an injection-resistance benchmark can still happily install a redirected package inside a wrapper that never surfaces the redirect.

The practical consequence is that you cannot buy install-time safety by selecting a safer model. A model leaderboard on injection resistance tells you little about whether your specific agent will run pip install on a typo’d name. The harness wrapper is a first-class security control, and most of the time it is not documented as one.

Are AGENTS.md and .cursorrules a privileged attack channel?

They are the most direct instance of the threat the paper defines, with one honest caveat: arXiv 2607.15143 measures package-install attacks delivered through README and dependency files. Its empirical data does not specifically test AGENTS.md or .cursorrules. The extension to those files is an analytical inference from the paper’s setup-instruction model, not a measured result.

That caveat matters less than it sounds. AGENTS.md and .cursorrules are files explicitly written for the agent to consume as instructions. A README the agent happens to parse is an incidental privileged input; an AGENTS.md is a deliberate one. The mechanism is identical: the agent reads a repo file as trusted context, then acts on it. If anything, instruction files are a stronger escalation path because they are designed to change agent behavior rather than merely describe the project. A README that lies about an install command is a bug-shaped attack; an AGENTS.md that rewrites the agent’s operating assumptions is a feature-shaped one.

This is also where the paper’s threat model diverges from the prompt-injection literature most teams already know. Prior work on gradient-based injection attack generation and SecAlign’s preference-optimization defense targets LLM-integrated applications where the attacker controls user-facing data. SecAlign, for instance, builds a preference dataset of secure versus insecure outputs and trains the model to prefer the legitimate instruction over injected content. That work is real and useful. But it assumes the instruction channel itself is clean. The repo-config case inverts that assumption: the attacker controls the instruction channel itself, the part the existing defenses assume is trusted. Security-aware prompt defenses do not transfer cleanly, for the same reason they only partially work in the install-time setting. They were trained against an attacker who poisons the data, not an attacker who authors the spec.

The second-order consequence the title is really asking about follows directly: the trust boundary for coding agents moves from the model to the repo filesystem. Security teams that were reasoning about model behavior now have to reason about which files the agent reads at setup, who wrote them, and whether the agent’s privilege level matches the file’s trust level.

How do MCP servers extend the attack surface?

MCP (Model Context Protocol) servers add a dynamic tool-invocation channel with their own taint-style vulnerabilities, and they compound the same trust problem the setup files create.

Research on MCP server security focuses on taint-style flows, where untrusted data from one tool’s output reaches another tool’s invocation, and on metadata-based attack surfaces, where the tool description itself becomes an injection vector. The same study finds that taint-style vulnerabilities make up a substantial fraction of MCP server flaws, require non-trivial code changes to fix, and are patched slowly by the community. The mitigation that work proposes, rewriting tool descriptions with embedded behavioral guidance, is the MCP analog of the security-oriented prompt in the install-time setting. It has the same limitation: it only closes the gap for the dimension the description explicitly names.

The connection to AGENTS.md is direct. A repo’s MCP server configuration registers servers whose descriptions the agent reads as trusted metadata. An attacker who controls the config controls both which tools the agent can call and how those tools are described. The description is, in effect, another instruction file that the agent honors without independent verification. Existing MCP server implementations, like the citecheck bibliographic-verification server, demonstrate practical tool-integration patterns, but the security landscape of MCP servers with respect to taint-style vulnerabilities remains underexplored. The ecosystem is still young enough that the attack patterns are being discovered faster than the mitigations are being standardized.

What defense stack actually stops this?

A deterministic pre-install check that verifies package names, sources, and versions before any install runs closes most of the gap. Security-oriented prompts close part of it, but only for the dimension they name.

The deterministic check is the load-bearing control. It means resolving package names against a known-good registry, enforcing a source allowlist, pinning versions, and checking them against published advisories before the package manager is invoked. None of this is novel. It is standard software-supply-chain hygiene, lockfiles, provenance, and allowlists, applied to the step where the agent decides what to install. The paper’s contribution is to show that this boring control is doing almost all of the work the model is getting credit for.

Why prompts underperform is structural. They are probabilistic, model-dependent, and scope-limited. A prompt that says watch for typosquats does not also catch registry redirection unless it is told to, and even then it inherits the harness-model fragility. Every dimension the prompt does not enumerate is a dimension the attacker gets for free. The paper measured exactly this: a security-oriented prompt recovers part of the gap on the dimension it names and leaves the rest.

The broader defense categories, sandboxing, instruction signing, repo-config allowlists, and prompt-isolation boundaries, are the right frame for thinking about the threat model. The caveat is that vendor claims about sandboxed execution need to be read against the privileged instruction channel. A sandbox that constrains what the agent can do on the host but still executes the contents of an attacker-controlled AGENTS.md is solving a different problem than the one this paper describes. Sandboxing the blast radius is not the same as vetting the input.

What does this mean for running agents against third-party repos?

The practical cost is that every repo you point an agent at is now untrusted code that executes through the agent. The review step that used to sit between “clone the repo” and “run the code” has to move earlier, to between “clone” and “let the agent read its own configuration.”

This changes the workflow for anyone letting agents operate against community or vendored repos. The config files the agent reads at setup, AGENTS.md, .cursorrules, MCP server registrations, lockfiles, need the same review treatment you would give a dependency before pinning it. Letting an agent auto-run setup against a freshly cloned third-party repo without that review is the same posture as running curl | bash from a stranger’s README, except the agent does it for you and logs less. The shared-filesystem assumption that most agent harnesses still default to means a malicious setup file can reach beyond the project directory if the agent’s permissions are not scoped.

The write-once, compromise-many property widens the blast radius. A malicious dependency compromises one install at a time. A malicious setup file is re-read and re-honored on every clone, by every agent, across every developer who touches the repo. For an internal monorepo this is manageable. For an open-source repo with external contributors, it is an ongoing supply-chain exposure that has to be governed, not assumed away.

What should teams do today?

Treat AGENTS.md, .cursorrules, and MCP server configs as untrusted code. Gate the agent’s install step behind a deterministic allowlist. Assume the model will not catch source-based attacks, because the data says it will not.

Concrete steps, in priority order:

  1. Run a deterministic pre-install check: registry allowlist, version pinning, advisory scanning, before any package install the agent triggers.
  2. Review repo-level agent configuration before the first agent run, and CI-gate changes to AGENTS.md, .cursorrules, and MCP config the same way you would gate a change to a Dockerfile.
  3. Scope the agent’s install privileges to what the repo actually needs. Do not run setup with elevated credentials against unvetted repos.
  4. Do not rely on security-prompted models to detect source-based or registry-redirection attacks. The paper measured near-universal failure here.
  5. When letting agents work on third-party repos, insert a review step before agent configuration is honored, and prefer repos whose agent config is signed or reviewed.

The strongest limitation to keep in mind is that the paper tests a specific set of harnesses and models, and detection rates vary by pairing. That variance is exactly why the deterministic check has to be the control you depend on. The model is a contributing factor; the allowlist is the guarantee.

Frequently Asked Questions

Do malicious setup files affect all ecosystems equally?

The paper directly tested npm and Cargo, where nearly every model failed on source-based registry redirection. Detection rates for separator-confusion names like azurecore versus azure-core varied by model-wrapper pairing. The paper does not report data for Python pip, Ruby gems, or Go modules, so the specific failure modes there are inferred, not measured.

Why do prompt injection defenses fail against repo-level attacks?

Prior defenses like SecAlign assume the instruction channel is trusted and only the data input is poisoned. Repo-config weaponization inverts this: the attacker controls the instruction file itself. Security-aware prompts that guard against untrusted data cannot close a gap where the attacker authors the spec the model follows.

What is the operational cost of deterministic pre-install checks?

Every package the agent proposes to install must be resolved against a known-good registry, version-pinned, and checked against advisories before the package manager runs. This adds seconds to minutes per install depending on registry latency. The paper finds this boring control does almost all the work that prompt-based defenses attempt, and it is cheaper to audit than a probabilistic model defense.

Do sandboxed execution environments block setup-file attacks?

A sandbox constrains what the agent can do on the host system, but it does not vet the inputs the agent reads. If the sandboxed agent still executes an attacker-controlled AGENTS.md, the attacker controls the agent’s decisions within the sandbox. Sandboxing limits blast radius, but it does not address the privileged input channel.

How will MCP servers change this threat landscape as they mature?

MCP servers expose tool descriptions as trusted metadata that the agent reads without verification. Research on taint-style vulnerabilities in MCP servers shows these flaws are patched slowly. As more repos ship MCP server configs, attackers gain a larger surface for tool-description injection, where a single compromised description can redirect how every agent that touches that repo invokes its tools.

sources · 5 cited