NetInjectBench: Prompt Injection Becomes a Network Availability Problem

NetInjectBench (arXiv:2607.10490), posted 11 July 2026, is a benchmark for indirect prompt injection against large language model agents that use network operations tools. The point of the paper is not the benchmark itself. It is the blast radius the benchmark exposes: when an agent with net-admin scope ingests untrusted content, injection stops being a data-exfiltration problem and becomes an availability and integrity problem on production gear.

What does NetInjectBench test, and why is network ops different?

The benchmark runs 130 scenarios¹ across three open-weight models, separating untrusted artifact text from trusted policy metadata so a defense can be tested on whether it blocks injected instructions without refusing legitimate work.

The scenarios break down as 40 benign, 40 weak-attack, 40 strong-attack, and 10 approved high-impact changes, each evaluated against Qwen2.5-7B, Llama3.1-8B, and Mistral-7B.¹ The tool use under evaluation is network operations, where an executed action lands on production infrastructure.

The distinction matters because the existing literature overwhelmingly treats the worst case as an exfiltrated API key or a scraped inbox.² Those are confidentiality losses, serious but recoverable. A net-admin agent that executes an injected command can reroute traffic, open a firewall hole, drain an IP address pool, or tear down a workload. The blast radius is availability and integrity on infrastructure that may not have a clean rollback, and the recovery path involves people with pagers, not a credential rotation.

The vector matters too. The injection rides on content the agent already reads as part of its job: tickets, alerts, logs, runbooks, and ChatOps messages. None of that is exotic traffic; it is the normal feed of a network operations center. Separating untrusted artifact text from trusted policy metadata is the benchmark’s core design move, because a defense that conflates the two either trusts everything the tool says or nothing.

How often do attacks succeed against each defense?

Naive execution, where an agent simply acts on what it reads, reached an 82.50% unsafe tool-action rate across 240 attack instances,¹ and the four prompt-level defenses the paper tested cut that figure without closing it.

Prompt-only safety reached 25.63%, Self-Reminder 21.67%, Spotlighting 18.33%, and a Two-Pass LLM Judge 10.00%.¹

Static allowlisting dropped the unsafe rate to 5.00%,¹ but at a cost the abstract states plainly: it blocked all approved changes, yielding 0.00% usefulness and 100.00% overblocking on the approved-change cases.¹ An allowlist that refuses every real change is less a security control than a kill switch with a configuration file attached.

The reason the prompt-level defenses plateau is structural. The injected text and the legitimate instruction travel the same channel, so any rule strong enough to block one risks blocking the other. Spotlighting and Self-Reminder try to demarcate untrusted content inside the prompt; the Two-Pass LLM Judge tries to re-read it with suspicion. The 10.00%¹ from the judge is the best any content-inspection approach in the paper manages, and it is still an order of magnitude too high for an availability control.

The metadata-aware policy gate is the result the paper leads with, and it carries an asterisk. Under the paper’s stated metadata-integrity assumption, it produced 0 of 240 unsafe attack actions, with a 95% Wilson upper bound of 1.58%,¹ while preserving 99.17% attack-scenario usefulness and 100.00% approved-change usefulness.¹ The load-bearing phrase is “under the stated metadata-integrity assumption.” The gate authorizes an action only if it carries trusted policy metadata, which is precisely the property real deployments are worst at upholding.

Why can’t prompt-filter vendors solve privileged tool use?

The prompt-injection market has formed around chat and document content, and the defenses it sells are tuned to that surface, where the worst case is a leaked secret and the remedy is content inspection, instruction-hierarchy prompts, input rewriting, or output redaction.

None of those are wrong, and none of them address an agent that holds an SSH key. The gap is the difference between blocking a bad message and blocking a bad action. A content filter can score a ChatOps message as suspicious and still lose, because by the time the filter runs the agent has been granted a session on the router. Once the tool is bound and the capability is live, the injection is not competing with a filter for attention; it is competing with an authorization boundary for whether the action runs at all.

NetInjectBench makes the gap concrete. The prompt-level defenses tested in the paper, Spotlighting, Self-Reminder, and a Two-Pass LLM Judge, left between 10% and 26%¹ of injected net-admin actions executable. That residual rate is tolerable in a summarizer and unacceptable in a firewall editor.

The deeper problem is that vendors optimize for recall against a corpus they can see: leaked prompts, known injection templates, red-team datasets. Network operations tools have no equivalent corpus. The set of legitimate actions is small, versioned, and audit-logged, but the set of dangerous ones is shaped by topology, change windows, and blast radius, none of which a generic content filter can evaluate. The information needed to authorize or refuse the action lives in the network, not in the prompt.

The practical test for any vendor selling “prompt injection protection” for an agent deployment is three questions. Does the product authorize the action, or only inspect the message? Does it read network-level metadata, or only prompt content? And does its published evaluation cover privileged tool use, or only chat and document scenarios? NetInjectBench does not survey shipping products, so it cannot answer the third question for any specific vendor. It does show that the prompt-level defenses drawn from the content-inspection literature leave a 10% to 26%¹ residual unsafe-action rate on privileged tools, which is the gap any product relying on those techniques inherits.

What are the three mitigation paths, and which tradeoffs do they force?

The benchmark does not prescribe an architecture, but its results point at three separable strategies: per-tool sandboxing, network-domain capability scoping, and air-gapping the agent from the control plane. Each trades something different.

Per-tool sandboxing isolates each tool behind its own policy and blast limit. An SSH tool can be scoped to a read-only user, a kubectl tool limited to a non-production cluster, an IPAM tool allowed to allocate but not to reclaim. The win is that a compromise in one tool does not cascade. The cost is operational: the sandbox has to be correct for every tool, the model has to know which tool to call for a given task, and someone has to maintain the mappings as the fleet changes. Sandboxing is necessary, but it decides how much damage an action can do once it runs, not whether the action is authorized.

Network-domain capability scoping is what the metadata-aware policy gate approximates. Authorization is evaluated against network-level metadata (who requested the change, whether it was approved, which change window it falls in, what topology it touches) rather than against the text of the ticket. This is the path that scored 0 of 240 unsafe actions in the paper,¹ and it is also the path that inherits the metadata-integrity assumption. If the policy metadata can be forged, tampered with, or desynchronized from the artifact text, the gate becomes a bypass target rather than a control. Real capability scoping means the metadata has to be signed, bound to a workflow, and impossible for the agent or its inputs to mutate. That is a systems-engineering problem, not a prompt problem.

Air-gapping the agent from the control plane is the bluntest option. The agent never holds a live tool; it proposes changes that a human or a separate, locked-down system applies. This reduces the injection threat to a recommendation-quality problem, at the cost of most of the automation that justified deploying the agent in the first place. It is the safe default and the option that makes the return-on-investment case hardest to defend.

What should network teams do now?

Start from the assumption that any content an agent reads is hostile until the action it requests is independently authorized, because that is the only posture the benchmark’s numbers support.

Concretely, bind network tools behind an execution-time authorization boundary rather than behind a prompt. Evaluate that boundary against the metadata a network already produces (change tickets, approval records, topology, blast radius) and treat the prompt-level defenses as defense in depth rather than as the control. If you cannot guarantee the integrity of the policy metadata the gate depends on, do not deploy the gate as if it were the 0% figure from the paper.¹

The shift the benchmark forces is organizational as much as technical. Prompt-injection defense has lived with AI safety teams and security vendors who tune for content. Network operations tools live with network engineers who have never had to treat an LLM as a privileged operator on their gear. The benchmark names the gap between those two groups. Closing it is the work that comes next.

Frequently Asked Questions

Do the NetInjectBench results apply to frontier models like GPT-5 or Claude Sonnet 5?

Not directly. The benchmark only tested Qwen2.5-7B, Llama3.1-8B, and Mistral-7B, so the 82.50% unsafe-action rate is a floor for smaller open-weight models, not a prediction for frontier models with stronger safety training. Even if the rate drops, the underlying risk stays the same: once an agent holds a live network tool, prompt hygiene cannot authorize or refuse an action on its own.

How does NetInjectBench differ from earlier prompt-injection benchmarks for web fetch or MCP tools?

Earlier benchmarks treated the worst case as data exfiltration through web fetch or MCP-style tools. NetInjectBench is the first to measure indirect prompt injection against network control-plane tools such as SSH, router and firewall configuration, kubectl, and IPAM interfaces, which turns the impact from a confidentiality loss into an availability or integrity incident on production infrastructure.

What is the smallest operational change a network team should make before deploying a net-admin agent?

Put every network tool behind an execution-time authorization gate that reads signed, workflow-bound metadata your CMDB or change system already produces, and run the agent in a read-only shadow mode first to log the actions it would have taken against real tickets. Treat prompt-level defenses such as Spotlighting or a Two-Pass LLM Judge only as defense in depth, because the benchmark showed they still let 10% to 26% of injected net-admin actions through.

Where can the metadata-aware policy gate fail in production?

It fails if an attacker can forge, tamper with, or desynchronize the trusted policy metadata from the artifact text, or if the metadata source itself is stale. A change ticket approved last week, a CMDB entry with stale topology, or a compromised approval workflow can all cause the gate to authorize the wrong action even though no prompt injection occurred.

What would force teams to abandon autonomous net-admin agents and air-gap them from the control plane?

A follow-up benchmark on frontier models showing prompt-level defenses still allow a 10% or higher unsafe-action rate, or a real-world demonstration that policy metadata can be forged in production, would make the automation case hard to defend. At that point, the only posture with a clean audit trail is to let the agent propose changes and have a separate human-reviewed system apply them.