Implement Context Compaction

The route

4 steps to Done

1. 01

   Add token accounting

   You cannot manage what the loop does not measure.

   Preview prompt + verify gate ▾ Hide ▴

   Instrument the agent loop with context accounting. Requirements: before every model call, compute the token count of the assembled context (system prompt + history + reminders) using the model's tokenizer or a reliable estimator; log it per turn with a breakdown (system, history, tool results, reminders); expose current usage as a fraction of the model's window; add a warning log at 60% and a trigger event at the compaction threshold (default 75%, configurable). Verify by running a file-heavy session and watching the per-turn accounting climb in the logs.

   • ✓ Count computed before every call
   • ✓ Breakdown by section logged
   • ✓ Threshold event fires at the configured fraction
2. 02

   Design the structured summary format

   Compaction quality is decided by the format, not the model.

   Preview prompt + verify gate ▾ Hide ▴

   Design the compaction summary as a structured document, not prose. Sections: TASK (the original request, verbatim), PLAN STATE (todo list snapshot with statuses), DECISIONS (choices made and why, one line each), WORKSPACE FACTS (files created/modified with one-line descriptions, key paths, commands that worked), CONSTRAINTS (rules and user preferences stated so far), OPEN ITEMS (unresolved questions, known issues), RECENT CONTEXT POINTER (what the kept-verbatim tail covers). Write the summarization prompt that produces this format from a transcript chunk, with explicit instructions: preserve exact file paths, exact names, and exact error strings; never invent; mark uncertainty. Test it on a saved long transcript and grade the output against the sections.

   • ✓ All seven sections defined
   • ✓ Prompt demands exact paths and names
   • ✓ Test summary graded against a real transcript
3. 03

   Implement the compaction operation

   Swap history for the digest without dropping the thread.

   Preview prompt + verify gate ▾ Hide ▴

   Implement compaction. When triggered: (1) split history into OLD (everything except the last N turns, default 10) and RECENT (kept verbatim); (2) run the summarization call on OLD with the structured prompt; (3) rebuild history as [summary-as-system-note] + RECENT; (4) re-render dynamic reminders (plan block) after the swap; (5) log before/after token counts and archive the pre-compaction history to a session file for audit. Guard rails: if the summarization call fails, retry once, then fall back to a cruder truncation that still preserves TASK and PLAN STATE; never compact mid-tool-execution - only between turns.

   • ✓ Recent tail kept verbatim
   • ✓ Pre-compaction history archived
   • ✓ Failure fallback still preserves task and plan
4. 04

   Prove continuity under compaction

   The only test that matters: does the agent still know what it is doing?

   Preview prompt + verify gate ▾ Hide ▴

   Run the continuity test suite. Design a task long enough to force at least one compaction (e.g. a multi-file refactor with verbose tool output). After compaction fires, probe the agent mid-session with: 'What was the original request?', 'Which files have you modified and why?', 'What did we decide about X?' (a decision from the compacted region). Verify answers match the archived pre-compaction history. Then let the task run to completion and confirm no work was redone and no constraint was forgotten. Log the full session with compaction events marked, and record token usage staying under the window throughout.

   • ✓ All three probes answered correctly
   • ✓ No redone work after compaction
   • ✓ Usage stays under the window to completion

Research-backed

Sources behind this flow