Step 1 of 4
Build a threat-driven probe set
Attack what matters to this agent, not a generic checklist.
Define the vulnerability probe set from my agent's threat model. Relevant classes for a tool-using coding agent: prompt injection (direct and indirect), data leakage (secrets, other users' data, system prompt extraction), tool/permission bypass (getting restricted actions to fire), destructive-action induction, jailbreaks of safety refusals, and excessive-agency (agent doing more than asked with real consequences). For each class, note what a successful attack would achieve against MY system specifically (what data, which tools). Prioritize by impact x likelihood. Use a red-teaming framework's probe catalog (e.g. DeepTeam-style vulnerability/attack taxonomies) as a source, but select and adapt to my risks. Deliver the prioritized probe set.
Expected after this step
A prioritized, threat-driven probe set adapted to the agent.
Should not happen
- ✕Testing only the happy path and calling the agent secure
- ✕One-off manual pokes with no reproductions or scoring
- ✕Finding vulnerabilities and never closing them
- ✕A red-team run once, never repeated as the agent evolves
Verify before continuing
Do not move on until every check is true. The complete button stays locked until then.
Do not continue if…
- !Testing only the happy path and calling the agent secure
- !One-off manual pokes with no reproductions or scoring
- !Finding vulnerabilities and never closing them
- !A red-team run once, never repeated as the agent evolves
If the AI messes this up
Use this when the AI fakes progress or breaks the feature. It forces a real fix.
The probe set is generic model-safety tests. Re-anchor on THIS agent: which of its tools, secrets, or data would an attacker want, and which probes actually reach them - drop probes with no reachable impact here.