Why the credential never enters model context

The obvious way to build an agent that acts inside a product is to hand the model the user's API key and let it call the API. It works in a demo. It is also a live credential inside a loop that reads untrusted web content, and that combination is the whole problem.

Our rule is short: the end user's credential never enters model context. Not in a system prompt, not in a tool argument, not in a log line. The model's job is to say what it wants done; something that is not a model decides whether it happens.

The broker

The model emits an intent: {tool: "reschedule_appointment", args: {id, starts_at}}. A separate service, the broker, receives it and runs a procedure in a fixed order: validate the arguments against the tool's schema (no extra properties, bounded sizes, safe path segments), check the allowlist (the tool is enabled on this site, its classification is within the plan, the session is identified and belongs to this user), require a confirmation for anything that writes, mint or fetch the credential, call the API, filter the response to the fields the customer allowed, and write the audit row. The model sees the filtered result and nothing else.

The order matters more than any single step. A confirmation the visitor gave is never consumed by a later refusal; nothing irreversible happens before every check has passed.

What it costs

Three things, honestly.

First, latency: an intent is a round trip to another service. We budget 1.5 seconds at the median for a tool round trip and stream "Checking your appointments…" while it runs, because beyond two seconds the perceived wait dominates.

Second, an integration step: the customer has to tell us how to obtain a credential for a user. The preferred answer is a token-exchange endpoint on their side that mints a short-lived, narrowly scoped token per session. The fallback is a stored service credential with an on-behalf-of header. Both are more work than pasting a key. Both are the reason the agent can be trusted with the keys to a clinic's calendar.

Third, a class of feature we cannot build: anything where the model would need to improvise a call the customer did not enable. That is not a loss.

What it buys

Prompt injection stops being a question of prompt engineering. A page the agent read can convince the model to suggest cancelling every appointment; it cannot make the broker do it, because the broker checks the classification, requires the confirmation card, and refuses destructive tools unless the customer switched them on. A successful injection produces a bad suggestion. That is a much smaller problem than a completed refund.

It also makes the audit log honest. Every row was written by the broker at the moment it did the thing, with the arguments it actually sent and the status it actually got, not by a model summarising what it believes it did.