Filtering model output was adequate when models only produced text. Once an agent can spend money and change state, authorisation has to move to the action.
Early guardrails filtered what the model said. That model is obsolete the moment your agent gains tools, because the risk is no longer an inappropriate sentence — it is an executed action with real consequences.
Treat the agent as an untrusted client of your API, because functionally that is exactly what it is. It receives instructions from strangers and turns them into calls against your systems.
Where the checks go
- Authorise at the tool boundary, in code, against the identity of the end user — never on the model's assurance that a call is appropriate
- Give each tool the narrowest possible scope; a tool that reads one collection is safer than one that runs arbitrary queries
- Rate limit per user, per session and globally — the global limit is what stops a prompt-injection loop becoming a five-figure invoice
- Make destructive actions require confirmation outside the model's control
- Log every tool invocation with its arguments; without that record you cannot investigate anything
Assume the prompt will be defeated
System prompts are guidance, not a security boundary. Anyone who has spent an afternoon with an agent has talked one past its instructions. Design so that a fully compromised prompt still cannot cause an action your authorisation layer would refuse — then the worst case is an embarrassing message rather than a breach.