Step 1 of 4
Build the secret detector
You can only redact what you can recognize.
Implement secret detection. Layers: KNOWN PATTERNS (regexes for common credential formats - cloud keys, tokens with recognizable prefixes, private key blocks, connection strings with embedded passwords, JWTs); CONFIGURED VALUES (the actual secret values the app knows about from its environment/secret store - these are exact-match redactable and catch anything the patterns miss); ENTROPY HEURISTIC (flag long high-entropy strings in suspicious contexts as a backstop, tuned to limit false positives). Package as a single scrub(text) -> redacted_text function returning text with secrets replaced by typed placeholders (e.g. [REDACTED:aws-key]). Unit-test against a fixture of real-shaped (fake) secrets and confirm no known format slips through.
Expected after this step
A layered scrub() function catching patterns, known values, and high-entropy strings.
Should not happen
- ✕Secrets in plaintext logs and traces, one breach from disaster
- ✕Redacting logs but leaving secrets in session records or memory files
- ✕Passing real credentials through the model context 'because it is convenient'
- ✕Assuming no leaks without ever scanning stored artifacts
Verify before continuing
Do not move on until every check is true. The complete button stays locked until then.
Do not continue if…
- !Secrets in plaintext logs and traces, one breach from disaster
- !Redacting logs but leaving secrets in session records or memory files
- !Passing real credentials through the model context 'because it is convenient'
- !Assuming no leaks without ever scanning stored artifacts
If the AI messes this up
Use this when the AI fakes progress or breaks the feature. It forces a real fix.
A real-shaped key slipped the detector. Add its format to the pattern set AND rely on configured-value matching as the safety net - the exact known secret should always be catchable regardless of format coverage.