Prompt-Cache-Aware Conversation Design
Structure your agent's context for prefix caching: stable prefixes, append-only history, and cache-friendly dynamic content.
The route
3 steps to Done
- 01
Audit context stability
Find every byte that changes between turns - each one is a cache killer.
Preview prompt + verify gate ▾ Hide ▴
Audit my agent's context assembly for cache hostility. Capture the exact assembled context for three consecutive turns and diff them. Classify every difference: EXPECTED GROWTH (new turns appended at the end - fine), PREFIX MUTATIONS (anything changing before the append point - cache killers), and VOLATILE INJECTIONS (timestamps, counters, randomized ordering, per-turn IDs inside stable sections). Common culprits from harness analyses: current-time in the system prompt, re-rendered plan blocks at the top, tool definitions re-serialized in unstable order. Produce the diff report with each mutation's source located in code.
- ✓ Diffs captured on real turns
- ✓ Every mutation traced to its code source
- ✓ Volatile injections separated from growth
- 02
Restructure for stable-first layout
The layout rule: immutable, then append-only, then volatile.
Preview prompt + verify gate ▾ Hide ▴
Restructure context assembly into three zones. ZONE 1 (immutable per session): system prompt with no timestamps or counters, tool definitions serialized in a fixed, sorted order. ZONE 2 (append-only): message history - new turns append; old turns are never edited, re-ordered, or re-annotated between compactions. ZONE 3 (volatile tail): dynamic reminders (plan block, current state, date if needed) rendered as the final messages, where they never break the prefix. Move every volatile item found in the audit into Zone 3 or remove it. Apply the provider's cache markers/breakpoints after Zone 1 and after Zone 2 where supported. Re-run the three-turn diff and verify zones 1-2 are byte-identical across turns.
- ✓ Timestamps and counters out of zones 1-2
- ✓ Tool defs serialized deterministically
- ✓ Post-restructure diff shows append-only behavior
- 03
Verify cache hits and measure savings
The API tells you the truth - read it.
Preview prompt + verify gate ▾ Hide ▴
Instrument and measure. Log the provider's cache-related usage fields from every response (cached/read tokens vs fresh/write tokens, per your provider's schema). Run a 10-turn session and chart per-turn: total input tokens, cached tokens, cost estimate. Expectations: turn 1 writes the cache; turns 2+ should show the bulk of input tokens as cached; a compaction resets the cache (log it as an expected event). Compare cost per turn against a pre-optimization baseline session of the same shape. Deliver the measurement table and the percentage savings. If cached ratios are low, check TTL behavior (calls spaced beyond the provider's cache lifetime) and breakpoint placement.
- ✓ Usage fields logged per turn
- ✓ Turn-2+ cached ratio meets expectations
- ✓ Savings quantified against baseline
Research-backed
Sources behind this flow
Tier 1 · claude-code-internals
claude-code-source-analysis (xiaonancs)
A systems-engineering reconstruction of the complete Claude Code agent harness from a reverse-engineered v2.1.88 snapshot (includes the extracted source zip plus appendix analyses). Part of a series that also covers OpenAI Codex CLI internals, giving a comparative harness view.
Tier 1 · claude-code-internals
Deep Dive: Multi-Agent Dispatch - Spawn, Fork, Direct
Source-level analysis of the three ways Claude Code dispatches work: Spawn (a named sub-agent with a fresh context, its own tool pool, permission mode, and model), Fork (a synthetic agent that inherits the parent conversation byte-for-byte so parallel workers share the prompt cache), and Direct (the main agent just does the task itself). Covers sync-to-background promotion via a promise race, recursion guards on forking, git-worktree isolation, and a decision matrix for picking a strategy per task.