Agent safety and governance is the practice of arranging an AI agent's authority so that one bad instruction cannot cause much harm. That instruction might come from a model that misread a request, from an email or document written to steer the agent, or from a retry that fires after an action already happened. None of those are fixed by a stronger model or a longer prompt, because each of them still ends with model output turning into an effect.
The core method I use is simple to state: the model proposes, deterministic tools act. The model's job is to turn fuzzy input into a structured proposal in a fixed shape, with quoted evidence and an allowed "unsure" answer. Narrow tools then decide whether that proposal becomes an action. Each tool checks the requester's scope and region, the current policy version, the live state of the record, date windows and limits, and refuses or escalates when anything is unclear.
Around that core sit a few rules that keep it honest. Retries must be harmless by design, with idempotency keys, an outbox, caps, time windows and one-step undo, so a lost reply followed by a retry still acts only once. An unknown effect never authorizes a second action. Every agent is checked for three legs, private data, untrusted content and a way out, and at least one is broken by design. Authority is scoped to the request rather than the session. Approvals have to survive changes in payload and context. And the agent is graded on the end state of the system, never on its own report that it is done.
The essays below take these pieces one at a time: where authority should live, how to keep it from leaking through alternate paths, what to do with lost replies, and how to tell whether a guard actually held.
Start here
- Build the tool broker around the authority that survives alternate paths
- Give each agent turn its own authority
- Unknown is an outcome: designing agent systems around lost replies
- The agent timed out after the change went through
- Make human approval survive changes in payload and context
- Your hook ran: did the forbidden action stay blocked?
- A safe AI widget is a small capability system
- Let an agent ask for a credential without ever seeing it
- Why agents cannot grade their own work
Questions this topic answers
- How does an agent decide which tool to call, and who decides whether it runs?
- How do you stop an agent from doing something it shouldn't, even when its input is hostile?
- How do you make retries and lost replies safe?
- Which of the three legs should you break for a given agent?
- How do you prove an agent action worked without trusting the agent's own report?
If you are working through how much authority your agents should hold, connect with me on LinkedIn to compare notes and discuss.