Prompt-Injection Defenses for Tool-Using Agents: Isolation, Allowlists, and Output Filters

1. Introduction

A language model that only generates text can be wrong, sycophantic, or leaked. A tool-using agent — a model bound to retrieval, mail, browsers, shells, or APIs under a shared authorization boundary — can also act. NIST’s Generative AI Profile (AI 600-1) treats that inventory as the system. The failure this paper cares about is not that the model said something it was told not to say. It is that the agent did something that endangers life, health, property, or the environment.

That is the Safe characteristic in AI RMF 1.0 (NIST AI 100-1), not the secure and resilient row. The two share machinery. Confidentiality of prompts and weights is a security outcome. An unauthorized wire, a deleted production volume, or a medical-device setting changed because retrieved text was treated as an instruction is a Safety outcome that happens to travel through a security-shaped hole.

Indirect prompt injection — instruction-like content arriving through retrieved pages, attachments, tool results, or memory, rather than from the seated user — is now a standard item in public taxonomies (Greshake et al., 2023; OWASP Top 10 for LLM Applications; NIST AI 100-2). This paper does not add a new attack. It compares three defenses that vendors already ship, and asks which Safety claims each one can honestly support.

Four observations structure the rest of the paper.

The model is not the perimeter. Once untrusted tokens share a context window with developer instructions, the model may treat those tokens as a request. A control that only asks the model not to is advisory.

The three defenses sit in different places. Isolation sits on channels and runtimes. Allowlists sit on what is mounted and what arguments are accepted. Output filters sit on text immediately before a tool call or a user-visible act.

They are not substitutes. A filter in front of an unbounded tool belt is a warning label. An allowlist of every product tool is an inventory, not a gate. Isolation that still holds production credentials is a labeled box around the same blast radius.

Chat-only scores are the wrong Measure. A refusal on a public jailbreak list does not say whether a tool side effect occurred. Safe is about harm in the world under defined conditions of use. A sentence detector is not that measurement.

Contributions. We (1) locate prompt injection on tool-using agents under Safe without absorbing Safety into Security; (2) give a layered account of isolation, allowlists, and output filters against a shared threat model; (3) state a composition order, worked cases, and a Measure that counts actions; (4) list claims that should be refused.

This paper is a comparison, not a bake-off. It does not rank named products and does not describe how to produce a steered context.

2. Background: why chat Safety is the wrong object

2.1 From answers to actions

The first wave of public “AI safety” evaluation for large language models was mostly about text: toxicity, disallowed advice, political sycophancy, and single-turn jailbreaks. Those problems are real. They are not the object of this paper. A model that emits a disallowed sentence has produced content. A model that is bound to tools has a second interface: a structured call that another process will execute.

NIST’s Safe row is written in the language of defined conditions of use and of harm to life, health, property, and the environment. It is closer to classical system safety — what the machine is allowed to do to the world — than to a content-moderation queue. AI 600-1 is the document that makes tools, retrieval, and memory first-class. Once those are in the inventory, an evaluation that never asks “did a side effect fire?” has measured a different characteristic.

2.2 Why Security programs pick up the file

Prompt injection looks like an integrity problem: untrusted input is treated as a command. Security teams already own input handling, sandboxing, and least privilege. That is the right machinery. It is the wrong filing if the program then reports only leaked prompts, jailbreak refusal rates, and red-team pass/fail on chat. A prior paper in this series warned against security synecdoche: equating AI RMF implementation with an adversarial robustness eval. The converse error is Safety synecdoche: calling a filter a Safety program because Safe is in the marketing title.

The two characteristics share a hole and split a harm. Confidentiality loss can cause Safety harm later. This paper’s Measure question is whether the agent acted.

2.3 Why three defenses, not twelve

Production agent platforms accumulate many controls: system-prompt policy, provider guardrail IDs, rate limits, spend admission, plan mode, human approval, logging. We group the ones that are actually argued as injection defenses into three families.

• Isolation answers “what can this confused model reach?”
• Allowlists answer “what actions exist?”
• Output filters answer “does this forthcoming text look like something we should pause?”

Refusal training, constitutional classifiers on the user turn, and provider-side abuse detection sit around this loop. They are not treated as a fourth family here, because they are properties of the model or of an upstream API, not of the agent’s tool boundary. A prior paper treated the pre-tool gate as the load-bearing object of a workspace trust layer. This paper asks what Safety claim each family of contents of that gate can carry.

3. Threat model

3.1 System

The system is an AI 600-1–shaped agent: a model, a system prompt, retrieved corpora, tools, and session or long-term memory, under one workspace authorization boundary. The seated user is trusted to ask. Content that arrives through retrieval, attachments, websites, mail, or prior tool output is not trusted to instruct.

We assume the attacker cannot mint workspace credentials and cannot edit the gateway’s policy object. They can place text where the agent will read it. They cannot, in this model, replace the allowlist compiler or the sandbox policy. If they can, the rest of the paper is moot — that is a compromised control plane, not prompt injection.

3.2 Trust boundaries

Four boundaries matter, and they are easy to collapse in a demo.

1. Seated user → model. The user’s ask is intentional. It is still not a request to disable Safety gates. Custom instructions that widen tools are a policy bug, not a feature of “helpfulness.”
2. World → model. Retrieved and attached text crosses into context. This is the usual injection surface.
3. Model → tools. A structured call leaves the model and enters a runtime that can change the world.
4. Tools → world. The runtime uses identities, networks, and data stores. Isolation lives here as much as in the context window.

A design that only hardens (2) and leaves (3) and (4) wide has built a content filter, not an agent Safety posture.

3.3 Harm we score

We score harmful action, not leaked chat. Examples of the class, not a catalog: sending mail that commits the organization; writing to a repository or ticket that changes production; transferring funds or placing an order; altering an environment that affects health or property.

We do not score “the model produced a disallowed sentence.” That is a content-policy problem and belongs in a different paper. We also do not score whether a human later copied a bad suggestion into a console. That is a different Map item (human-in-the-loop as the actuator). If the product’s tool is the actuator, the agent owns the harm.

3.4 What “the model is steered” means

The worst case for a Safety designer is not a cute single-turn jailbreak. It is a context in which the model’s next tool call matches the attacker’s intent and still looks locally reasonable to a schema checker. Defenses that hold in that case are load-bearing. Defenses that hold only when the model remains mostly aligned are advisory.

Multi-turn and memory make this worse without requiring a new attack family. A label stripped on write, a summary that drops a “untrusted” tag, or a later turn that treats yesterday’s tool output as developer voice will steer a model that refused the first page. Isolation that does not apply to memory is isolation of the first hop only.

This paper does not describe how to produce that context.

3.5 Out of scope

We do not treat weight exfiltration, training-data extraction, or provider-account takeover. We do not treat physical robots except as a reminder that Safe was written with life and environment in mind: a browser agent and a device agent differ in actuators, not in the logic of the three defenses. We do not treat open-weight fine-tunes that strip refusal; that is a model-supply problem. We do not claim a complete incident taxonomy for Manage.

4. Isolation

4.1 Two halves that are often conflated

Isolation has two halves.

Context isolation keeps untrusted text out of privileged instruction channels. Retrieved pages, attachments, and tool results are marked, quoted, or routed through a reader that cannot rewrite the system prompt. Dual-model patterns — a quarantined reader that may see the page, a privileged planner that may not — belong here.

Runtime isolation privilege-separates the tool process: a locked-down browser profile, a container without production credentials, a network policy that cannot reach payroll. The model may still request a harmful call. The runtime cannot complete every call.

Teams that ship only the first half have a formatting convention. Teams that ship only the second have a sandbox around an agent that still treats the page as policy. Safety wants both.

4.2 Context isolation in practice

Useful forms, from weaker to stronger:

• Markup. Untrusted spans are wrapped or role-tagged so a well-behaved model can tell them apart. This is documentation for the model. It is not a gate.
• Channel split. Developer policy, user ask, and retrieved text enter the request through different API roles or different messages that the provider treats as non-equivalent. This is better, and still fails if the planner is later handed a concatenated “brief.”
• Quarantined reader. A model that may see the page produces a structured extract (facts, quotes, uncertainty). A planner that may call tools never sees the raw page. The extract schema is the remaining injection surface; keep it narrow and non-imperative.
• No-memory of raw untrusted text. The page is not written into long-term memory as if it were a user preference or a standing instruction.

The honest claim for context isolation is: instruction-like text is less likely to be treated as developer policy. The dishonest claim is: the agent cannot be injected.

4.3 Runtime isolation in practice

Useful forms:

• Identity split. Tool credentials are not the user’s production admin token and not the workspace billing owner. They are a scoped identity issued for this session or this mode.
• Network split. The tool runtime cannot reach networks that hold payroll, clinical, or control-system assets unless that reach is the named purpose of the workspace.
• Filesystem and browser split. A browser tool uses a fresh profile, no shared cookies with the operator’s day job. A code tool writes to a scratch tree, not the production checkout, until a separate promote step.
• Time and spend split. Admission controls (rate, budget) are not Safety by themselves, but they bound how much a steered loop can do before a human notices.

The honest claim is: unreachable property stays unreachable in this session. A sandboxed mail send still sends.

4.4 Memory as a third isolation surface

Session memory and long-term memory are how a one-hop defense dies on turn six. A Safety design treats memory writes as a tool: they have a schema, they carry a provenance tag, and they are not a back door into the system prompt. Summaries that drop provenance are a context-isolation failure delayed.

4.5 What isolation can claim

If context isolation holds, untrusted text is less likely to set policy. If runtime isolation holds, a steered model has a smaller world: fewer identities, fewer networks, fewer durable writes. That is a real Safety reduction. Property that cannot be reached cannot be destroyed by this agent in this session.

Isolation does not decide which remaining writes are acceptable. It is a blast-radius control, not an intent control.

4.6 How isolation fails

The planner still sees the page. The user asked a question “about this document,” so the privileged model is handed the raw text “just this once.”

Production identity in the box. The container is “isolated” and still carries the deploy key, because that was the fastest way to demo a useful write.

Pass-through tools. A “generic HTTP” or “run this command” tool is a hole in the runtime wall with a friendly name.

Memory laundering. Untrusted text is stored, stripped of tags, and replayed as standing guidance.

UX pressure. Isolation that makes the agent less useful will be disabled per session unless it is workspace-authoritative — the same Govern point as the trust-layer paper.

5. Allowlists

5.1 What it is

An allowlist is a fail-closed inventory of capability: which tools are mounted for this workspace, role, or mode; which hosts, mail domains, or repositories those tools may touch; which argument shapes are accepted. Plan modes that permit research reads and deny mutating tools are allowlists by another name. A prior paper in this series treated a pre-tool gate as the load-bearing object of a workspace trust layer. This paper treats the contents of that gate as the Safety question.

5.2 Layers

Four layers are easy to flatten into one boolean, “tools on.”

A product that mounts every verb and then “allowlists” string arguments has implemented a type checker. A product that mounts mail and allowlists * destinations has implemented a send button.

5.3 Who writes the list

Govern lives here. Owner or Admin writes the mount set. Members inherit it. The model does not mount tools. Custom instructions do not widen the list. A CISO who cannot read the current mount set does not have Measure.

Modes are not severity sliders. A read-only mode and a write mode are different residual action sets, like the trust layer’s standard and strict — profiles, not “more Safety.” Shipping one list that contains everything, then asking a filter to “be careful,” is the filter-only error with extra steps.

5.4 What allowlists can claim

Allowlists are the only one of the three defenses that still hold when the model is fully steered. A tool that is not mounted cannot fire. An argument that fails the schema cannot fire. That is why allowlists are the first control in a Safety composition, not the last. They answer “what can this agent possibly do to the world?”

That claim is only as strong as the list. An honest coverage strip says which verbs are mounted, not that “the agent is safe.”

5.5 How allowlists fail

Too wide. A list that contains shell, unrestricted browser, and generic HTTP is an inventory of the product, not a Safety control.

Schema-valid harm. A send to a finance alias can satisfy a mail schema and still move property. Allowlists decide existence, not meaning.

Who writes the list. If the model can mount tools, or if custom instructions can widen the list, the allowlist is advisory again.

Confused deputy at the other end. An allowlisted “open ticket” tool that a downstream human treats as already authorized relocates the harm; it does not remove it.

Shadow tools. Plugins, MCP servers, or user-installed connectors that bypass the workspace list are a second product pretending to be the first.

Stale lists. A destination that was safe last quarter (a test inbox, a staging cluster) is not automatically safe this quarter. Allowlists need owners and review, or they become folklore.

6. Output filters

6.1 What it is

An output filter inspects model text or proposed tool arguments after generation and before the side effect. Implementations range from pattern matchers, through classifiers, to a second model asked whether the forthcoming call violates policy. Some filters also sit on user-visible answers. This paper cares about the pre-tool filter, because that is where Safety lives.

6.2 Kinds

Matchers. Fixed phrases, regular patterns, or denylisted destinations. Cheap, explainable, and brittle. Useful as a tripwire for the cases you already named.

Classifiers. A trained or prompted detector that scores “instruction-like” or “policy-violating.” Broader than a matcher, still a text function. The score is not a measure of property harm.

Second model. An inspector model reads the proposed call (and sometimes the same untrusted page) and returns allow, block, or ask. This is scalable oversight in miniature. It inherits prompt injection at one remove if it sees the raw page. It should see the structured call and the policy, not the attacker’s prose, whenever that is enough to decide.

Rewrite versus ask. Silent rewrite (“we fixed your tool call”) hides the decision and trains users to ignore the boundary. Ask — stop, show a short reason, require a human — is the Safety-shaped path. A prior paper required same-request events without prompt bodies; the same rule applies here. A filter that fires and writes nothing did not happen for Measure.

6.3 Placement

The filter belongs on the tool-call object, after the allowlist has already rejected unknown verbs, and before the runtime uses credentials. Filtering the chat reply while letting the tool call through inspects the apology, not the action. Filtering the user turn is a different control (input policy). It does not substitute for a pre-tool look.

6.4 What filters can claim

Filters are a last look. They can catch known instruction-like framing, obvious policy phrases, or a short list of forbidden destinations that escaped the allowlist. They are useful as a tripwire and as an ask path. They are a poor sole control.

The honest claim is: known bad text can be paused. The dishonest claim is: we detect injection.

6.5 How filters fail

Paraphrase. A filter that keys on fixed phrases does not bound a steered model that can say the same thing another way.

The filter is a model. If the inspector is itself a language model reading the same untrusted text, it inherits the original problem at one remove.

Latency and fail-open. Filters that time out and allow the call are not Safety devices. Filters that fail closed and block the product will be turned off unless the allowlist already made most calls cheap to approve.

Wrong object. Filtering the chat reply while letting the tool call through inspects the apology, not the action.

Wrong metric. Catch-rate on a public jailbreak list is a detector score. It is not a count of prevented side effects.

Operator fatigue. An ask on every read trains people to click through. Asks should be reserved for residual writes after the allowlist did its job.

7. Comparison

Three readings follow.

First, filters are not a boundary. They do not change the capability set. Measuring filter catch-rate on a public jailbreak list is not a Measure of Safe. It is a Measure of how often a particular text detector fired on a particular list.

Second, allowlists without isolation still concentrate credentials. A narrow tool list running with production identity is a small door in front of a large vault. Isolation without an allowlist is a large door into a smaller vault. Safety wants both: few doors, small vault.

Third, UX is part of the control. A composition that makes ordinary research painful will be turned off. That is why the order matters: a small mount set makes isolation cheaper and makes filters rare enough that ask is still meaningful.

8. Composition and worked cases

8.1 Load-bearing order

1. Allowlists — decide the residual action set. Prefer read-only defaults. Mount writes per workspace and per mode, not because the model asked.
2. Isolation — decide what that residual set can touch. Separate untrusted content from policy text. Tag memory. Run tools without production credentials unless a named write requires them.
3. Filters — inspect the residual calls. Prefer ask over silent rewrite. Fail closed when the inspector is unavailable.

A prior paper argued that prompt-only policy packs are not a trust layer. This paper adds a Safety corollary: a filter-only pack in front of a full tool belt is the same error with a classifier attached.

8.2 Worked case: research workspace

A workspace is meant to read the public web and draft notes. The allowlist mounts a browser and a notes write; it does not mount shell, mail, or deploy. Isolation gives the browser a fresh profile and keeps retrieved pages out of the system-prompt channel; notes writes are tagged as user-authored, not as policy. A filter asks only if a notes write tries to include instruction-like standing rules. The residual Safety question is small: can a notes file later be treated as a system prompt? That is a memory-isolation item, not a reason to mount a shell.

8.3 Worked case: mail-capable workspace

A workspace mounts mail because the job is correspondence. The allowlist names destination domains. Isolation uses a send-only identity that cannot read the executive inbox and cannot touch billing. The filter asks a human on first send to a new domain that day. The remaining Safety question is whether send should be in this workspace at all, and whether schema-valid mail to an allowed domain can still commit the organization. Those are Govern and Map questions. The three defenses can only make the residual smaller.

8.4 Worked case: filter-only product

A workspace mounts every tool the product ships. Isolation is a shared production token in a container. A classifier scores each chat reply. This is the failure mode the paper is written against. The classifier may post an impressive catch-rate. The agent can still act.

8.5 Coverage

A workspace that ships the composition may say it has reduced action risk from untrusted text. It should not say it has implemented Safe, measured health or environmental harm, or “solved prompt injection.” Neighboring characteristics stay off the strip unless they are actually enforced: fairness, explainability, and the rest of the RMF are not implied by a pre-tool gate.

9. Measure: count actions, not sentences

Govern without Measure is a policy PDF. The trust-layer paper required same-request, aspect-tagged events without prompt bodies. This paper specifies what those events must be able to answer for Safe.

A reviewer should be able to filter, for a workspace and a window:

• How many tool calls were blocked by the allowlist (unknown verb, bad destination, bad schema)?
• How many were asked (filter or policy) and then allowed or denied by a human?
• How many writes completed, by tool and destination class?
• How many isolation faults were recorded (missing provenance tag, sandbox denied, credential scope miss)?

They should not need the prompt body to see that picture. They should not be shown a jailbreak-leaderboard percentage as a substitute.

What this Measure still is not: a count of injuries, a clinical outcome, or an environmental impact. Safe in the RMF sense is broader than “we blocked some sends.” Honesty is to say “we counted residual actions under this mount set,” not “we measured Safe.”

10. Failure modes to refuse

We observe recurring category errors (a typology, not a statistical sample):

• Filter-only Safety. A full tool belt plus a sentence detector, advertised as a Safety program.
• Shared-credential isolation. A box whose identity is production.
• Product-inventory allowlist. Every shipped tool is mounted; the list is a catalog.
• Jailbreak synecdoche. Refusal rates stand in for Safe.
• Prompt-pack Safety. System-prompt text that asks the model not to be injected, with no gate.
• Fail-open inspector. Timeouts allow the call.
• Apology filter. The chat is filtered; the tool is not.
• Per-session opt-out. Members disable the composition for a “quick task.”
• Seven-characteristic theatre. A pre-tool gate is advertised as the AI RMF.

The last is the same certification theatre the mapping paper warned against. ISO/IEC 42001 remains the certifiable management system. This composition is a Manage posture for one harm class.

11. Related work

Public taxonomies. Greshake et al. (2023) described indirect prompt injection into LLM-integrated applications. OWASP’s LLM Top 10 lists prompt injection as a first-class risk. NIST AI 100-2 places prompt injection among adversarial ML techniques; AI 600-1 names tools, retrieval, and memory as system components. Weidinger et al. taxonomize language-model risks beyond the adversarial setting; Amodei et al. framed accident risk that is not primarily an attacker problem. Safe is closer to the accident framing than to a red-team scoreboard, even when the hole is adversarial. This paper uses that inventory; it does not extend the taxonomy.

Agent gates. Industry practice already includes plan modes, tool-permission strips, and provider-side guardrails. The workspace trust-layer paper in this series argued that those pieces need a policy object and an evidence plane. Here the question is which Safety claim each piece can carry.

Classical safety and least privilege. Sandboxing, capability attenuation, and fail-closed defaults are old. The novelty is not the primitives. The novelty is that the “program” choosing the syscall is a stochastic model reading untrusted prose. Isolation and allowlists are how that fact is made unsurprising.

Alignment and oversight. Debate, critique, and other scalable-oversight patterns try to catch bad intent when the model is stronger than the reviewer. They are complementary and unmeasured here. A filter that is a second model is a cheap, weak form of that idea and should be labeled as such.

We do not treat refusal training as a fourth defense. Refusal is a property of the model. This paper’s object is the agent loop around the model. A more aligned model is good and still does not revoke a mounted shell.

12. What this paper does not do

We did not run a bake-off. There is no table of catch-rates, no named vendor ranking, and no claim that one architecture dominates on a public benchmark. Published injection evaluations are still dominated by single-turn text tests that do not score whether a tool side effect occurred.

We did not produce a control catalog. SP 800-53 and forthcoming AI overlays remain the place for numbered technical controls. We did not claim ISO/IEC 42001 alignment or EU AI Act conformity. We did not measure whether workspaces that ship this composition differ in incident rates from those that do not.

The paper is therefore a comparative contribution. Its failure mode is the usual one for comparisons: sounding more complete than an untested composition.

13. Discussion

13.1 Procurement language

“Supplier must prevent prompt injection” is not a clause. A more precise clause names the mount set, the isolation properties (identity, network, memory provenance), the fail-closed behavior of any inspector, and the events a reviewer can query. Binary “injection-proof” language recreates the “NIST compliant” error from the mapping paper.

13.2 When a filter is the right first ship

Early products ship a detector because it is visible. That is a reasonable tripwire if the allowlist is already narrow. It is an unreasonable program if it is the only control. The test is simple: if the detector is removed, what can the agent still do? If the answer is “everything the product can do,” the program was the detector.

13.3 Open-weight and local agents

Nothing in the three-family comparison requires a hosted model. A local agent with a wide tool belt and no gate is the same Safety object. Isolation may be easier on a machine the operator owns (a VM they already use) and harder (the agent is on the same desktop as the deploy keys). The claims do not change.

13.4 Revision risk

AI 100-1 is under revision. The Safe wording may move. The structural claim we care about — action harm is not a jailbreak score; the three defenses are not substitutes — does not depend on a subcategory number.

14. Conclusion

Prompt injection on a tool-using agent is a Safety problem when the failure is an action that can endanger life, health, property, or the environment. It is also a security-shaped hole. Filing it only under Security invites a program that measures leaked prompts and jailbreak refusals, then mounts a shell.

Isolation, allowlists, and output filters are the three defenses this paper compared.

• Isolation shrinks the world a confused model can touch, including memory, not only the first retrieved page.
• Allowlists shrink the actions that exist at all — tool, destination, schema, and mode — and still hold when the model is steered.
• Filters inspect residual text, and do not revoke tools.

Used in that order, with events that count blocks, asks, and completed writes, they are a posture. Used as synonyms — especially “we have a filter” as a synonym for Safe — they are a label on an open door.

Acknowledgments

This paper is a comparison of defensive patterns, written as a sequel to a mapping of the NIST AI RMF suite and a description of a workspace trust layer. It is not affiliated with NIST.

@online{violet2026injectiondefenses,
  author = {Haigh, John},
  title = {Prompt-Injection Defenses for Tool-Using Agents: Isolation, Allowlists, and Output Filters},
  date = {2026-09-14},
  year = {2026},
  url = {https://www.violetai.ca/en/research/prompt-injection-defenses/paper},
}