Skip to content
Flows
Library Paths Vault Generate Progress Run Demo Flow
Agent QA & Security Intermediate · 60-90 minutes

Secrets Hygiene in Agent Logs

Keep credentials out of prompts, tool outputs, logs, and traces: detection, redaction, and safe handling across the whole agent pipeline.

Start Route · 4 steps

The route

4 steps to Done

  1. 01

    Build the secret detector

    You can only redact what you can recognize.

    Preview prompt + verify gate ▾

    Implement secret detection. Layers: KNOWN PATTERNS (regexes for common credential formats - cloud keys, tokens with recognizable prefixes, private key blocks, connection strings with embedded passwords, JWTs); CONFIGURED VALUES (the actual secret values the app knows about from its environment/secret store - these are exact-match redactable and catch anything the patterns miss); ENTROPY HEURISTIC (flag long high-entropy strings in suspicious contexts as a backstop, tuned to limit false positives). Package as a single scrub(text) -> redacted_text function returning text with secrets replaced by typed placeholders (e.g. [REDACTED:aws-key]). Unit-test against a fixture of real-shaped (fake) secrets and confirm no known format slips through.

    • ✓ Common credential formats covered
    • ✓ Configured actual values exact-matched
    • ✓ Placeholders typed for later auditing
  2. 02

    Redact at every capture point

    One unscrubbed sink undoes all the others.

    Preview prompt + verify gate ▾

    Apply redaction everywhere content is persisted or surfaced. Route through scrub() at: log emission (wrap the logger), trace/span attribute capture (the observability scrubber), session record writes, memory writes, error/exception reports (stack traces and locals can carry secrets), and any content shown in the UI beyond the live session. Critically, also scrub MODEL-FACING content where the agent does not need the raw value: tool results containing credentials get redacted before entering context (the agent reasons fine about '[REDACTED:api-key]'). Enforce via a single choke: capture functions must accept only pre-scrubbed content, or call scrub internally. Audit the codebase for any direct writes to these sinks that bypass the wrapper.

    • ✓ All sinks route through scrub()
    • ✓ Model-facing tool output scrubbed where raw secrets are unneeded
    • ✓ Bypass paths audited and closed
  3. 03

    Inject real secrets safely

    The credential must do its job without ever touching the LLM or disk logs.

    Preview prompt + verify gate ▾

    Implement safe secret injection for tools that genuinely need real credentials (calling an authenticated API, connecting to a database). Pattern: the harness holds a secret reference (name), resolves the real value from a secure source (env/secret manager) inside the tool handler at execution time, uses it, and returns only redacted results; the real value never enters the model's context, the tool arguments the model produces, logs, traces, or storage. The model passes a reference ('use the github token'), not the secret. For any flow where a secret would otherwise transit the model, redesign it around references. Test: exercise a credential-using tool and grep every artifact (context, logs, traces, session) for the real value - it must appear in none.

    • ✓ Secrets resolved inside handlers at execution time
    • ✓ Model passes references, not values
    • ✓ Grep of all artifacts finds no real secret
  4. 04

    Scan and prove no leaks

    Trust the hygiene only after hunting for what escaped it.

    Preview prompt + verify gate ▾

    Run the leak-hunt and lock it in. TEST SESSION: run a deliberately secret-heavy session - a tool that returns an API key, a command that echoes an env var, an error that includes a connection string, a user message pasting a token. AFTER: run a secret scanner (a dedicated tool or your scrub() in detection mode) across every stored artifact: log files, trace store, session records, memory files, error reports, and any temp files. Zero real secrets is the bar. For each leak found, fix the missed capture point and re-run. Then add a CI/pre-commit secret scan over the repo and a scheduled scan over runtime artifact directories, so a future leak is caught automatically. Document the redaction guarantees and the (rare) places raw secrets legitimately live.

    • ✓ All artifact types scanned
    • ✓ Every found leak traced to a capture point and fixed
    • ✓ CI/scheduled secret scanning in place

Research-backed

Sources behind this flow

Tier 5 · mcp-tooling

Rulebound

Deterministic guardrails for AI coding agents: policy-as-code that runs the deterministic part of code review (AST checks, regex, diff analysis) against agent plans, diffs, and evidence. Its .rulebound/rules/ and docs/threat-model/ are directly reusable for agent QA.

Tier 6 · agent-research

OpenLIT

OpenTelemetry-native observability for LLM apps and agents: one-line instrumentation for traces, metrics, cost tracking, plus prompt management and a playground - GenAI observability on open standards.