Tracking Prompt Injection in LLM Agent Pipelines: Where to Place Canaries

If an agent reads email, web pages, or documents, a safe-looking final answer can hide an injected instruction carried into its memory. You need to know which boundary the payload crossed before deciding what to contain.

Kill-Chain Canaries measured 950 runs across five production LLMs (Wang, MIT, and Zhang, University of Chicago; arXiv:2603.28013). It planted a unique canary token in every injected payload and recorded how far each token traveled. That stage-level measurement supports a blunt conclusion: you cannot reliably block prompt injection at ingestion, and defenses that claim to do so keep failing in controlled evaluations.

The workable strategy is containment with forensics: plant a canary token on every untrusted input channel, log its progress, and gate the action corresponding to the furthest stage it reaches. That trace helps decide where to spend detection effort.

One ASR number hides where your pipeline failed

Most prompt-injection reporting reduces to attack success rate: did the agent do the bad thing, yes or no. The preprint argues this conflates two very different events. In the paper’s example, a downstream agent that scores 0% ASR might be safe because the model refused the injected instruction, or because an upstream summarizer silently dropped the injected text before it ever arrived. Those two pipelines need opposite responses, and outcome-only scoring cannot tell them apart. This is the same observation-versus-action boundary that earlier work drew in definitions: per StruQ’s framing, an injection only succeeds when the response obeys the hidden instruction rather than treating it as data.

The preprint’s Figure 1 makes the problem concrete. In a three-boundary PDF relay experiment, outcome scoring rated both Claude Haiku 4.5 and GPT-4o-mini at 0/3. The canary trace showed they were failing differently: Haiku’s token never reached the memory write in 3/3 runs, while GPT-4o-mini’s token propagated to shared memory and was read by the second agent in 3/3 runs, without being executed. The canary was absent from one pipeline’s memory write and present in the other’s, where it crossed an agent boundary. The same outcome score hid different marker-propagation paths.

The propagation figures need to stay tied to their test setting. They come from a single preprint’s benchmark harness (agent_bench, 950 runs across five models, batches dated March 10-27 and April 1, 2026), not production stacks. The defense-evaluation figures later come from the separate papers that measured them. Copy the measurement method; do not expect those propagation numbers to be your baseline.

Four checkpoints, one token

The method has two parts. Give every injected payload, or every untrusted document in production, a unique canary token. Then have a logger record the furthest of four checkpoints it reaches:

1. Exposed: the token appears in a tool result.
2. Persisted: the token appears in a memory write.
3. Relayed: a second agent reads the token.
4. Executed: the token appears in an outbound tool call.

The preprint tested five production LLMs (GPT-4o-mini, GPT-5-mini, DeepSeek Chat v3, Claude Haiku 4.5, Claude Sonnet 4.5, at temperature 0.0) across six attack surfaces: web page, pre-seeded memory, tool result, visible PDF text, white PDF text, and audio transcript. That surface list doubles as your ingestion instrumentation checklist. If your agent reads a channel on that list, that channel gets a canary.

Each checkpoint calls for a different containment decision. Logging only the final run outcome loses that distinction.

Mapping checkpoints onto real boundaries

Place the checkpoints where content changes hands in a typical agent deployment:

Use the table as a proposed response plan. The stage-to-gate mapping is inference from the preprint’s checkpoint design and established least-privilege practice; the paper measures propagation but does not deploy or validate these gates. The execution-boundary actions align with what classifier-based defense work recommends: give the model read-only or minimum-necessary access, rate-limit it, and use a classifier to log inputs before the LLM sees them.

The retrieval boundary deserves its own note. Work on retrieval-augmented pipelines treats the retriever itself as part of the security perimeter, with countermeasures like neutralizing instruction-like spans in retrieved contexts and pruning retrieval results to shrink the attack budget, as surveyed in a clinical prompt-injection benchmark. A Persisted canary hit is exactly the signal that tells you those countermeasures failed upstream, which is information a final-output monitor cannot give you.

What a canary hit proves, and what it does not

A canary hit shows that the marker reached a stage. It does not prove the model acted on the injected instruction. GPT-4o-mini’s token reaching shared memory and Agent B in the relay experiment shows the marker crossing two boundaries; the attacker’s goal was still not executed. Conversely, a clean outcome does not certify containment, since the payload may have been stored and await a later trigger the benchmark did not exercise.

A missing marker does not prove the instruction was removed either. A summary could retain a paraphrased instruction while dropping the token, as the preprint’s limitations explain. Even an Executed hit can be a false alarm if the model quotes the token while refusing the attack. The authors did not rule that confound out across all runs. Check what was carried and what action occurred before treating the trace as a containment verdict.

The inverted mechanism already exists in the literature. A known-answer detection method described in Gosmar and Dahl’s framework paper signs sensitive instructions within command segments issued by authorized users, and the secret key’s absence from output signals that an injected instruction overrode the task, detecting compromise without filtering inputs or outputs. The kill-chain approach generalizes this from one signal at the output to a trace across the pipeline. Both share the same honesty: they tell you where the payload is, not what the model intended.

The trace changes the responder’s job. “canary reached Persisted, never Relayed” points to a memory store to scrub. “canary reached Executed” points to a live containment problem. Groundy has covered the adjacent failure where injection becomes an availability problem on production gear. Finding the stage is what tells you which kind of incident you have.

Defenses guard only what they inspect, and sometimes not even that

The lightweight defenses in the preprint failed against non-adaptive attacks. Some failed because they did not inspect the channel carrying the attack: pi_detector and write_filter missed those channels. Spotlighting also failed on content it did inspect, with the payload inside its delimiters in the text relay and tool_poison scenarios. The write_filter blocked the PDF relay but not the text relay. None of these failures required an adaptive adversary.

Coverage explains the first two failures; spotlighting’s is worse, because the payload was marked as untrusted inside its delimiters and the attacked models followed it anyway. Independent work confirms the pattern. Spotlighting mitigates indirect injection by delimiting, marking, or encoding untrusted input so provenance stays salient, which presupposes the content is wrapped. Cross-tool attack research shows tool descriptions are attack surfaces where spotlighting is ineffective by design, and where prompt-injection detectors classify most malicious outputs as Safe. Detector selection is its own minefield: in UniGuardian’s evaluation, only UniGuardian, LLM-based detection, and Granite-Guardian reliably separated benign from malicious inputs, while Prompt-Guard-86M and perplexity-based detection performed poorly. Even training-time defenses carry placement caveats: SecAlign reports 0% ASR against the ignore attack across five open-weight models, but its authors flag RAG and agent settings, where injected prompts hide inside long documents mixed with genuine data, as the harder real-world case.

The conclusion for placement: a filter or detector is a per-channel control, so instrumentation must be per-channel too. Static filters fail in exactly the ways Groundy’s earlier coverage of context-aware defenses documented: the attack only works because attacker-controlled evidence is present at decision time, and a filter that never inspects that evidence channel sees nothing.

Model choice changes propagation, not just ASR

The Haiku versus GPT-4o-mini divergence carries a procurement implication. Two models with identical outcome scores had different marker-propagation paths: one left the token out of its memory write, while the other carried it into shared state. The preprint also ran 186 PDF relay runs split between same-model pairs (138) and cross-model pairs (48), because relay behavior depends on the pairing, not just the model. If your architecture relays between agents, the propagation profile of the pair is the unit you need to measure, and you need canaries in your own harness to measure it, because these five-model results do not transfer.

Measure the pairing and channels you actually use. Propagation depends on your pipeline, models, and inputs; vendor numbers cannot settle that profile. You learn it by running tagged payloads through the real pipeline.

Blind spots the canary cannot cover

Write these limits into the incident plan. Three surfaces sit outside text-canary reach:

• Tool descriptions. Cross-tool harvesting and polluting attacks operate on the descriptions agents read when selecting tools, not on content channels you can tag. Detectors return Safe on these outputs by design.
• Pixel-level attacks. WebInject optimizes stealthy perturbations to rendered webpage pixels that induce web agents to perform attacker-specified actions. There is no text payload for a canary to ride.
• Adaptive adversaries. The preprint cites Nasr et al., in which adaptive attacks bypassed all 12 defenses evaluated, most above 90% attack success. A canary regime assumes the payload survives in recognizable form; an attacker who knows the tagging scheme will strip or mutate tokens. Gates must assume evasion.

For the pixel-level and tool-description attacks there is no text payload, so a canary cannot fire even at the Executed checkpoint. The control that still applies is the execution-boundary gate rather than the canary itself: the attacker’s action has to materialize as a tool call regardless of how the instruction arrived, and approval or hard-blocking at that boundary does not depend on a tagged token. That is a reason to make the Executed gate the strongest control in the stack, not the weakest.

Rewrite the incident-response plan around stage localization

For each untrusted channel your agent reads, plant a unique canary token, log the furthest of the four checkpoints, and decide the gate action in advance. Strip or flag at Exposed. Quarantine at Persisted, since a memory write containing a live payload is a stored attack that can fire on later retrievals. Require human approval at Relayed, because a second agent reading the token adds another agent’s tool permissions to the blast radius. Hard-block at Executed and freeze the session for forensics.

The incident plan needs to answer “which stage did the payload survive to” before choosing whether to scrub a document, purge a memory store, or rotate credentials the agent touched.

Keep two limits attached to the framework. All propagation rates and per-model differences here come from one preprint’s benchmark and await independent corroboration. There is fresh reason for caution: a multi-agent incident-response preprint was withdrawn after a code audit found its scorer reading a source constant rather than model output. Canaries also provide containment and forensics instrumentation, not prevention.

For a first, low-cost hardening step before the next deployment, I would add the memory-write hook: reject or quarantine any write containing an untrusted-origin token. It turns the most dangerous silent failure, persistence, into an alert.