Step 1 of 4
Extract behavior into versioned artifacts
You can only track what has an identity.
Extract every behavior-bearing artifact into versioned files. Inventory: system prompt sections, tool descriptions and schemas, permission/danger rules, routing tables, reminder templates, repair prompts, sub-agent prompts. Move each into a dedicated files structure (prompts/, tools/, rules/ or equivalent) loaded at startup - no behavioral string literals left in code. Define the harness version: a semver bumped on behavioral change, or the git SHA of the artifacts tree; implement version resolution at startup and stamp it into every session record and log line header. Verify by changing one file and seeing the new version in the next session's logs.
Expected after this step
Versioned artifact files with the harness version stamped into sessions.
Should not happen
- ✕Prompts as unversioned strings buried in code, edited live
- ✕Changelogs listing file edits instead of behavior changes
- ✕No baseline eval, so behavioral drift is discovered by angry users
- ✕Sessions unattributable to harness versions when debugging regressions
Verify before continuing
Do not move on until every check is true. The complete button stays locked until then.
Do not continue if…
- !Prompts as unversioned strings buried in code, edited live
- !Changelogs listing file edits instead of behavior changes
- !No baseline eval, so behavioral drift is discovered by angry users
- !Sessions unattributable to harness versions when debugging regressions
If the AI messes this up
Use this when the AI fakes progress or breaks the feature. It forces a real fix.
Strings keep sneaking back into code. Add a lint/CI check greping for prompt-like literals in source (long strings near model calls) - make the easy path the compliant path.