Indirect Prompt Injection Benchmark: Which LLM Agent Surfaces to Audit First

Forty Docker-executed tasks, five payload categories, twelve tracked runtime surfaces: that is the concrete inventory behind INFUSE (arXiv:2610.20776), a self-reported preprint from Vanderbilt University that claims to be the first benchmark uniting execution realism, attack-surface coverage, and signal isolation for indirect prompt injection. Its practical value for a platform or security engineer is not a leaderboard. It is a checklist scaffold: a documented enumeration of every place untrusted content can reach an agent, which is exactly the list most teams have never written down.

The decision this article addresses is where to spend a limited injection-testing budget. The answer the evidence supports is: enumerate before you test. Inventory every channel where external content reaches the model, map each channel to a deterministic control, and treat any vendor injection-resistance claim as unverified unless it names the surfaces it was tested against. What the evidence does not support is a ranked list of “most exploitable surfaces,” because INFUSE reports no per-surface attack-success rates in its main results. The audit order below is a prioritization heuristic built from the taxonomy plus a standard agent architecture, not a measured vulnerability ordering. That distinction matters, and I will keep returning to it.

What INFUSE actually is

INFUSE evaluates coding-style agents in a full execution loop: 40 long-horizon tasks in real Docker sandboxes (self-reported), with agent tooling for shell access, code execution, package management, networking, and version control. Payloads are planted in five targeting categories: file names, file content, config files, terminal output, and archives. Observations are tracked across 12 read-side runtime surface types, a figure that is also self-reported from the same preprint.

Two properties separate it from the browser-only predecessors it critiques. First, surface breadth: INFUSE positions WASP and WAInjectBench as “restricted to the DOM and rendered page content, omitting the file, config, terminal, and build-artifact surfaces that drive real agent workflows.” Second, signal isolation. The authors argue that concurrent agentic safety benchmarks “conflate indirect injection with other failure modes, making it impossible to attribute a vulnerability to the channel that produced it.” Signal isolation is the property that makes surface-level prioritization conceivable at all: if you cannot tell which channel carried the payload, you cannot decide which channel to fix first.

The preprint’s example trace is concrete enough to be useful in a design review: a malicious comment in a training script instructs the agent to upload a private SSH key, resulting in key exfiltration. File content goes in, a credential goes out. The preprint also reports per-model utility scores under attack versus benign execution across 18 models (self-reported; Claude-3-Haiku is omitted from the benign baseline). As cited inside INFUSE, Dziemian et al. (2026) found attacks succeeding against all 13 models tested across 41 realistic agentic scenarios, yielding over 8,000 successful exploits. Treat that as a secondhand citation within an unreplicated preprint, not an established base rate.

Mapping the taxonomy onto your agent

The taxonomy becomes actionable when you lay it over a standard agent architecture: model, system prompt, tool definitions, retrieval and file readers, shell or code execution, network calls, and MCP or third-party tool connections. Every arrow carrying content the operator did not write is a surface. INFUSE’s five payload categories and twelve runtime surfaces name the file-and-execution side of that diagram; Microsoft’s registry documentation and LLMail-Inject extend it to the enterprise side.

The rows come from documented sources rather than intuition: the first five trace to INFUSE’s payload categories and runtime surfaces, the web and email rows to both INFUSE’s environment description and the LLMail-Inject challenge, and the MCP row to Microsoft’s agent-registry documentation, which warns that “some of your information and data (such as authentication keys and prompt content) may be passed to the Third-Party Tool.”

One nuance worth stating plainly: this table does not tell you which row is most likely to be exploited in your stack. It tells you which rows exist. Coverage is what the benchmark supplies; ranking is what it does not.

The control column: what “deterministic” buys

The reason the checklist maps each surface to a deterministic control rather than a model-side filter is that the counter-evidence against filters is unusually direct. The llm-security-test-lab repository codifies the OWASP GenAI LLM Top 10 2026 posture that “LLM01 treats filters and system prompts as partial controls and emphasizes structural validation, least privilege, and deterministic mediation.” Its mechanism is provenance separation: “user input and external content are checked separately,” so an instruction found inside a file can never inherit the authority of an instruction typed by the operator.

The lab’s paired-path numbers are worth citing with their caveat attached: baseline exploited 13 of 13 OWASP 2026 cases, protected path exploited 0 of 13, with 0 of 20 benign controls blocked and a 0.00% false-positive rate, all self-reported by the repository’s author. The numbers demonstrate that the architecture is implementable, not that it transfers to your stack; the corpus and the controls were built by the same hands.

The static-filter side of the story is even more cautionary. The ai-security-platform project self-reports a 96.2% detection rate (25/26, measured 2026-10-22 at commit 33500d7) for its Rust injection-detection engine, and then concedes in its own documentation: “The corpus is small and was written alongside the signatures, so these figures show that the CI gate holds, not how the engine performs on attacks it has never seen.” Its MCP tool-call firewall, with schema validation, least-privilege allow-lists, output validation, and escalating blast-radius containment, is marked planned rather than built. That is an honest snapshot of where the community control layer actually sits in October 2026: the architecture is right, the coverage is unproven, and the MCP surface specifically is mostly still on the roadmap.

Why template benchmarks mislead, and what vendor claims omit

The strongest independent evidence is LLMail-Inject (arXiv:2506.09956), a dataset from a realistic adaptive injection challenge. Its findings bound every assurance discussed above. Meta SecAlign achieves under 5% attack success rate on previous benchmarks (self-reported within the paper), but the paper notes those benchmarks “rely on 100–1,000 template-based attacks with limited diversity,” and defenses tuned to them remain vulnerable to adaptive strategies including social engineering and persuasion. In the challenge itself, no submission solved the GPT-4o-mini sub-levels with all stacked defenses in place (self-reported). The same paper documents that Microsoft issued CVE-2025-32711 for an email-based prompt injection affecting production infrastructure, which settles any remaining question about whether this is a lab problem.

The control direction LLMail-Inject argues for is contextual detection: reasoning about who an instruction is addressed to, what actions align with the task, and whether the action is what the user actually asked for. That is a different discipline from pattern-matching known payloads, and it aligns with the deterministic-mediation posture rather than the filter posture.

This is also where vendor injection-resistance claims fall short. A claim like “under 5% ASR” is surface-blind: it says nothing about whether the evaluation covered your file readers, your terminal stream, your MCP outputs, or only a template corpus. The practical rule I would apply in procurement or design review is simple. If a resistance claim does not name the surfaces tested, treat it as unverified. INFUSE’s taxonomy gives you the vocabulary to demand that scoping; a vendor that cannot map its claim onto file, config, terminal, archive, network, and tool-output surfaces has not answered the question you are actually asking.

Enumerate the estate before testing any of it

Enumeration is not the same as testing, and Microsoft’s own documentation is the clearest illustration. The agent registry in the Microsoft 365 admin center “provides a centralized view of all agents available for your organization,” enumerating agents by channel surface (Copilot, Teams, Outlook, Microsoft 365 apps, SharePoint) and consolidating agent-related risk signals into one admin view (Microsoft’s documentation). That is the inventory step, and for many tenants it will be the first time the full agent estate appears in one place.

Microsoft is also unusually candid about the tool-output surface: “MCP implementations are susceptible to risks, such as attacks, cascading failures, and loss of human oversight.” The recommended mitigations are vetting MCP servers, approval mechanisms, and monitoring cascading behavior. None of that injects anything. A registry tells you what exists; it does not tell you what survives a hostile README. Teams that stop at enumeration have done the cheapest part of the work.

The governance wrapper for turning the inventory into a review process comes from the ai-security-governance-framework, which bridges NIST AI RMF and the OWASP LLM Top 10 into SOC deployment checklists. Its framing is worth borrowing verbatim for an audit charter: “Autonomous AI action is a privilege earned by the environment, not a default granted by the vendor. Permission levels are enforced at the agent level, not the policy level.” That principle is what makes deny-by-default tool policy more than a slogan: the agent’s permissions, not the policy document, are the enforcement point.

An audit order under a real budget

Putting the pieces together, here is the order I would work in, and the reasoning is traceable rather than measured:

1. Enumerate. Registry or equivalent inventory of every agent, its channel surfaces, its tool and MCP connections, and its permission tier. Microsoft’s registry covers the first two for M365 tenants; the rest is on you.
2. Map surfaces. For each agent, list every channel where untrusted content reaches the model: file names and contents, config files, terminal output, archives, repository metadata, network responses, web and email context, MCP tool outputs. The INFUSE taxonomy is the reference list for the execution-side rows.
3. Place deterministic controls where they can sit. Provenance separation on ingest, canonicalization and schema validation for structured content, deny-by-default tool policy, output gates on tool results before they re-enter context. These are the controls with the strongest mechanistic case in the sources above.
4. Then test, surface by surface. Work the five INFUSE payload categories against your own stack. Bespoke per-app testing is not optional yet; nothing in the evidence shows a unified benchmark can replace it.

I would put MCP tool outputs and file content at the top of tier one, but I want to be precise about why: not because any source measured them as most exploitable, but because Microsoft’s own documentation flags key and prompt-content leakage to third-party tools, because INFUSE’s exfiltration example runs through file content, and because the MCP tool-call firewall in the ai-security-platform project is still planned rather than built. Surfaces that are high-consequence, vendor-flagged, and least defended are a defensible first tier. A measured ranking would be better, and it does not exist yet.

What one preprint cannot tell you

The strongest limitation deserves its own paragraph rather than a trailing caveat. Everything above hinges on a single self-reported preprint with no independent replication as of 2026-10-24. The preprint establishes the taxonomy, the 40-task and 19-model scope, and the SSH-key example; the paper reports no per-surface attack-success rates in its main results, and the tested frameworks, surfaces, and defense baselines are unverified. The lab and detection-engine numbers are self-measured on self-authored corpora, and their authors say so. LLMail-Inject is the most independent evidence here, and its message is deflationary: defenses that score well on template benchmarks still fall to adaptive attacks, and no stacked defense cleared the hardest challenge levels.

So the honest verdict has two halves. INFUSE’s taxonomy is immediately useful as an enumeration standard, and “enumerate every surface, then place deterministic controls, then test” is a process improvement most agent teams can adopt this quarter regardless of how the benchmark’s numbers hold up. But a unified surface benchmark does not yet replace bespoke per-app injection testing, no vendor resistance claim should be accepted without surface scoping, and any article, including this one, that ranked your surfaces by exploitability would be fabricating the part that matters most. Enumerate first. The taxonomy is the contribution; the ranking is still unwritten.

Frequently Asked Questions

What are the five payload categories used in the INFUSE benchmark?

Payloads are planted in five targeting categories: file names, file content, config files, terminal output, and archives.

How should vendor injection-resistance claims be evaluated?

If a resistance claim does not name the surfaces tested, treat it as unverified.