Memory Consolidation and Pruning Schedule

The route

4 steps to Done

1. 01

   Split memory into layers

   Different lifetimes deserve different homes.

   Preview prompt + verify gate ▾ Hide ▴

   Restructure the memory store into two layers. WORKING: entries from recent sessions (default: last 14 days or last 5 sessions), loaded eagerly, expected to be noisy. LONG-TERM: consolidated entries with provenance, loaded via index/recall. Migrate the existing store: entries confirmed more than once or user-stated go to long-term; the rest goes to working with their dates. Update write paths (new observations land in working; direct user preferences may go straight to long-term) and read paths (session start loads working fully plus the long-term index). Verify layer counts and loading behavior after migration.

   • ✓ Migration rules applied and logged
   • ✓ New writes route by evidence type
   • ✓ Session loading reflects the layer split
2. 02

   Implement the consolidation run

   Promotion with merging is where memory earns its keep.

   Preview prompt + verify gate ▾ Hide ▴

   Build the consolidation job (triggered at session end or on a schedule). Steps: (1) gather working entries past the promotion threshold (re-observed, or referenced in a successful task); (2) for each, search long-term for related entries; (3) merge: combine overlapping semantics into a single best phrasing, keep the union of unique facts, update provenance to cite both sources, supersede the old versions with links; (4) promote the merged entries to long-term; (5) expire working entries past TTL that never earned promotion; (6) write a consolidation report (promoted, merged, expired counts with samples). Run it against a store seeded with deliberate duplicates and verify the merge preserved every unique fact.

   • ✓ Promotion threshold is evidence-based
   • ✓ Merges cite both sources
   • ✓ Unique-fact preservation verified
3. 03

   Add decay scoring and safe pruning

   Forgetting is a feature when it has a paper trail.

   Preview prompt + verify gate ▾ Hide ▴

   Implement decay and pruning. Decay score per long-term entry from: age since last_confirmed, recall frequency (from the recall log), and contradiction events (each hit lowers it). Below a threshold, entries become prune candidates - EXCEPT protected classes (user preferences, security-relevant warnings) which never auto-prune. Pruning moves candidates to an archive file with full metadata; a restore command brings them back by ID or search; hard deletion only after 90 days in archive. Run the pruning job on the store and review the candidate list by hand once to calibrate the threshold before trusting it.

   • ✓ Decay uses recall-log data
   • ✓ Protected classes exempt
   • ✓ Restore-from-archive works by ID and search
4. 04

   Track health over simulated months

   Prove the lifecycle keeps the store good, not just small.

   Preview prompt + verify gate ▾ Hide ▴

   Build health metrics and run the longevity simulation. Metrics (computed per consolidation run): total entries per layer, duplicate rate (near-duplicate detection sample), average entry age, recall precision on a fixed probe set (10 standing queries with known-correct answers). Simulation: script 8 weeks of daily sessions (scripted writes and recalls with realistic noise), running the full lifecycle throughout. Success: working layer size stays bounded, duplicate rate trends down, probe precision holds or improves, and the archive contains only justified prunes. Deliver the metrics timeline and a one-paragraph verdict on lifecycle health.

   • ✓ Probe-set precision tracked across runs
   • ✓ Working layer stays bounded
   • ✓ Verdict backed by the timeline data

Research-backed

Sources behind this flow