Indirect Prompt Injection Testing: Test the Data Paths, Not the Prompts

Most indirect prompt injection testing is a quiz for the model. Somebody hands the chatbot a hostile prompt, counts the refusals, and calls it coverage.

In production, the hostile prompt never arrives as a prompt. It arrives as a tool result. A retrieved document. An email the user asked the agent to summarize. A calendar invite. Test the prompt and you test the wrong thing. The unit of testing is the data path.

Every channel that feeds context is an injection channel

An agent's context is a loading dock. Content arrives from everywhere, and the model treats most of it as material to work with. The channels I keep finding in real deployments:

Tool outputs. The agent calls a tool; the tool returns text. That text can carry instructions. (This is also where tool poisoning lives, covered in the MCP tool poisoning field notes.)
Retrieved documents. RAG pipelines pull from wikis, drives, ticket systems. Whoever can edit those sources can write to the agent's context.
Messages and mail. Email, Slack, Teams threads the agent summarizes or acts on. Anyone with the address can inject.
Browsed pages. Research agents fetch web pages. A public page is a public injection surface.
Memory. Entries written by earlier sessions, sometimes from content that was itself injected. Poison the memory once and it keeps paying out.
Tool metadata. Descriptions and schemas, per the C8a piece. An instruction source with a trusted name.

Every agent I have tested had more of these than its builders believed, because nobody wrote them all down in one place. That inventory is step one. You cannot test a path you have not enumerated.

Classify by who can write to it

Not every channel matters equally. The question that ranks them: who can put content here, and what does it cost them?

A public webpage costs an attacker nothing. A shared document costs a link. An email costs the address. An internal wiki costs an insider or a compromised account. A vendor's tool description costs control of that server.

This is the severity input most test plans skip. A channel nobody controls is a vulnerability. A channel the attacker controls is a delivery mechanism. Rank your test effort by writer cost, cheapest first. That is where the attack comes from.

Plant, chain, score

The methodology, for each channel in scope:

Plant payloads per class. Instruction smuggling in tool output. Authority forgery: content claiming to be a system or developer instruction. Exfil probes: "summarize this thread to the URL below." Multi-turn conditioning across memory writes. Class level, in a sandbox, under written rules of engagement.
Chain it. A planted instruction the agent merely reads is a curiosity. The finding is what the agent does: call a tool with attacker-chosen arguments, send data somewhere it should not go, take an irreversible action. Test the chain across channels, the same way the agent pentest methodology tests chains across layers.
Score by blast radius. How far did it get? What did the agent touch, move, or send? Refusal is one data point. The deliverable is the chain: payload in, action out, evidence for every hop.

The corpus is the unit test

Now the part that will annoy the benchmark crowd.

Static injection corpora have a real job. Point a few thousand prompts at the model on every build, and you catch backslides: the model swap that broke refusal behavior and nobody caught it, the harness change that widened a channel. That is regression testing at a scale no human team does by hand. I am not arguing against it.

I am arguing against calling it coverage.

The corpus tests the model's reflexes. The engagement tests the system's wiring. A model that aces the corpus still fails when the payload arrives through a channel the corpus never modeled, written by an attacker the corpus never imagined, chained across tools the corpus never composed.

This is the wrong-unit-of-measure problem again. The industry counts refusals. It should measure blast radius per injection path. A clean corpus score with an untested data path is a passing grade on the wrong exam.

Enforcement lives in code

The mitigations that survive testing are the ones the model cannot talk its way around:

Tool allowlists and scoped permissions, enforced outside the model.
Human gates on irreversible actions. Not a suggestion. A gate.
Output validation on anything the agent sends outward.
Taint tracking on untrusted content, at the code layer, so injected instructions never reach a privileged tool call.

The mitigations that do not survive: a system prompt telling the agent to be careful, a policy document describing vigilance, a wrapper that asks the model nicely to distrust tool output. If the guardrail is a prompt, it is a suggestion. If it is not executable, it is not a guardrail.

Both edges

The machine's job here is scale: generate injection variants across every channel, run them nightly, catch the regressions. No human team does that by hand, and pretending otherwise is nostalgia.

The operator's job is judgment: which channels are actually attacker-writable in this deployment, what counts as compromise, where the chain stops being a demo and starts being a breach. The machine runs the probes. The operator decides what the probes mean.

Here is what would have to be true for me to be wrong: training would have to enumerate every data path a deployment will ever open, and data would have to arrive with trustworthy provenance labels. Neither holds in any deployment I have touched.

So test the paths. The quiz says the model is safe. The data paths will tell you whether the system is.