Skip to content
Flows
Library Paths Vault Generate Progress Run Demo Flow
Agent QA & Security Advanced · 120-180 minutes

Red-Team Your Agent with Adversarial Prompts

Systematically attack your own agent using red-teaming frameworks: vulnerability probes, attack methods, and a repeatable adversarial suite.

Start Route · 4 steps

The route

4 steps to Done

  1. 01

    Build a threat-driven probe set

    Attack what matters to this agent, not a generic checklist.

    Preview prompt + verify gate ▾

    Define the vulnerability probe set from my agent's threat model. Relevant classes for a tool-using coding agent: prompt injection (direct and indirect), data leakage (secrets, other users' data, system prompt extraction), tool/permission bypass (getting restricted actions to fire), destructive-action induction, jailbreaks of safety refusals, and excessive-agency (agent doing more than asked with real consequences). For each class, note what a successful attack would achieve against MY system specifically (what data, which tools). Prioritize by impact x likelihood. Use a red-teaming framework's probe catalog (e.g. DeepTeam-style vulnerability/attack taxonomies) as a source, but select and adapt to my risks. Deliver the prioritized probe set.

    • ✓ Probe classes mapped to concrete assets and tools
    • ✓ Prioritized by impact x likelihood
    • ✓ Framework catalog adapted, not copied blindly
  2. 02

    Arm each probe with attack methods

    One payload proves little; enhancement is where real attacks live.

    Preview prompt + verify gate ▾

    Expand each vulnerability probe into multiple attack methods. Techniques from red-teaming practice: DIRECT (the naive ask), OBFUSCATION (encoding, translation, leetspeak, splitting the payload), ROLE-PLAY/PERSONA (fictional framing, 'you are DAN'-style), CONTEXT MANIPULATION (fake system messages, delimiter escaping), MULTI-TURN (benign build-up then the ask, or priming across turns), and INDIRECT (payload delivered via a fetched document or tool result rather than the chat). For each vulnerability, author 3-5 concrete attacks spanning these methods, each with a clear success criterion (what observable outcome means the agent was compromised). Store attacks as data (id, vulnerability, method, payload/setup, success criterion) so execution can be automated.

    • ✓ Multiple methods per vulnerability
    • ✓ Indirect and multi-turn attacks included
    • ✓ Success criteria are observable
  3. 03

    Automate execution and score

    Repeatability turns poking into a program.

    Preview prompt + verify gate ▾

    Build the attack runner. For each attack: set up the scenario (including seeding untrusted documents/tool results for indirect attacks), run it against the agent in an isolated test environment (sandbox, disposable data), and evaluate the success criterion - automatically where possible (did the restricted tool fire? did a secret appear in output? did the destructive action occur?), with a calibrated judge only for fuzzy criteria. Score: attack success rate per vulnerability class, ranked by severity. Produce a report with per-attack pass/fail and, for every success, a reproduction (exact inputs and observed compromise). Run the full suite for a baseline and triage the findings by severity.

    • ✓ Indirect attacks seed their untrusted content
    • ✓ Success evaluated on observable outcomes
    • ✓ Every success has a reproduction
  4. 04

    Remediate and institutionalize

    A red-team is worth nothing until findings become fixes and regressions.

    Preview prompt + verify gate ▾

    Close the loop. REMEDIATION: for each finding by severity, implement a structural fix (an authority gate, an isolation improvement, a blast-radius limit, a refusal hardening) rather than a payload-specific patch; add the reproducing attack as a regression test asserting the fix holds. Re-run the full suite after fixes to catch fixes that regress other defenses. PROGRAM: wire the suite into the release process - it runs on every release candidate and blocks on any high-severity success; schedule a periodic expansion where new attack techniques are added as they emerge. Document the current security posture honestly, including known residual risks the suite has not closed. Prove the loop by fixing the top finding end to end.

    • ✓ Fixes are structural, not payload-specific
    • ✓ Each finding has a regression test
    • ✓ Suite blocks releases on high-severity regressions

Research-backed

Sources behind this flow

Tier 6 · agent-research

DeepTeam

An LLM red-teaming framework (from the DeepEval team): 40+ vulnerability probes and 10+ attack methods (prompt injection, jailbreaking, leakage) to adversarially test LLM apps and agents.