Memory Recall: Ranking and Relevance
Make recall surface the right memories: hybrid lexical+semantic search, rank fusion, and explainable retrieval.
The route
4 steps to Done
- 01
Build the lexical layer
Exact matches are the floor of trustworthy recall.
Preview prompt + verify gate ▾ Hide ▴
Implement lexical search over memory entries. Requirements: tokenize entries and queries (lowercase, split identifiers on common separators so auth_handler matches 'auth handler'); score with BM25 (use SQLite FTS5 or a small library - do not hand-roll unless trivial); support quoted exact-phrase matching for error strings; return top-K with matched terms highlighted per result. Test with queries that MUST work lexically: an exact file name, an exact error message substring, and a specific command flag - all should hit their entries at rank 1.
- ✓ Identifier-aware tokenization
- ✓ Quoted phrases match exactly
- ✓ Exact-name tests hit at rank 1
- 02
Add the semantic layer
Paraphrase reach without surrendering to it.
Preview prompt + verify gate ▾ Hide ▴
Add embedding-based recall. Requirements: embed each memory entry (content plus topic keywords) at write time and store vectors alongside entries (a simple array store or SQLite sidecar is fine at this scale); embed queries at recall time; score by cosine similarity, return top-K. Batch-embed the existing store as a migration. Test with paraphrase queries: 'how do we deploy this' should find the entry about the release script even though it shares no keywords; 'login problems' should find auth-related failure memories. Record where semantic results disagree with lexical - those cases justify fusion.
- ✓ Vectors written at memory-write time
- ✓ Existing store migrated
- ✓ Paraphrase tests pass
- 03
Fuse rankings with boosts
Two imperfect rankers, one good one.
Preview prompt + verify gate ▾ Hide ▴
Implement rank fusion. Use reciprocal rank fusion: for each result, score = sum over methods of 1/(k + rank_in_method) with k=60 default; then apply multiplicative boosts: recency (entries confirmed in the last 30 days x1.2), confidence (high x1.2, low x0.8), and access frequency (frequently-recalled entries get a small boost, capped). Return fused top-K (default 5) with per-result score breakdowns retained. Build a small labeled test set (10 queries with known-correct memories) and report precision@3 for lexical-only, semantic-only, and fused - fused must match or beat both.
- ✓ RRF implemented with boosts
- ✓ Score breakdowns retained
- ✓ Precision@3 comparison reported
- 04
Make retrieval explainable
When recall misfires, you need to see why in one glance.
Preview prompt + verify gate ▾ Hide ▴
Add explainability and integrate with the agent. Each recalled memory returns: content, metadata, and an explanation line - e.g. 'matched: [deploy, script] (bm25 rank 2) + semantic 0.81 (rank 1); boosts: recency x1.2 -> final rank 1'. Integrate recall into the agent as a memory_search tool whose results render these explanations, and log every recall (query, results, explanations) to a recall log. Finish with a misfire drill: find one query from real usage where the wrong memory ranks first, read its explanation, identify the cause (tokenization, boost, embedding), fix it, and show the corrected ranking.
- ✓ Explanations show terms, scores, and boosts
- ✓ Recall log captures every query
- ✓ One misfire fixed via its explanation
Research-backed
Sources behind this flow
Tier 3 · memory
semantic-memory (RecursiveIntell)
Local-first hybrid semantic search backed by authoritative SQLite state plus a vector sidecar: facts, chunked documents, conversation messages, and episodes searched via BM25 (FTS5) and vector retrieval fused with Reciprocal Rank Fusion, with explainable search.
Tier 3 · memory
Memosynth Lite
A lightweight memory ingestion pipeline that processes JSON memory logs through summarization and deduplication, then fans out to three backends: a vector DB (Qdrant), a relational timeline (DuckDB), and a memory graph (Neo4j).