Step 1 of 4
Profile spend by call class
Find where the money actually goes before moving anything.
Profile my agent's model spend. Instrument every model call with: call class (main-loop reasoning, compaction summary, memory extraction, classification/routing, repair, sub-agent, embedding), model, tokens in/out, and computed cost. Run a representative day of tasks (or replay recent transcripts through the accounting). Produce the spend breakdown: cost share per class, average call size per class, and the top-3 cost concentrations. Typical finding worth checking: auxiliary calls (summaries, classifications) making up a large cost share while needing far less capability than they get.
Expected after this step
A spend breakdown by call class with concentrations identified.
Should not happen
- ✕Premium models summarizing tool output at 20x the needed cost
- ✕Cheap models silently degrading main-task quality with no eval to notice
- ✕Routing logic duplicated across call sites, drifting independently
- ✕No escalation, so cheap-tier failures just become task failures
Verify before continuing
Do not move on until every check is true. The complete button stays locked until then.
Do not continue if…
- !Premium models summarizing tool output at 20x the needed cost
- !Cheap models silently degrading main-task quality with no eval to notice
- !Routing logic duplicated across call sites, drifting independently
- !No escalation, so cheap-tier failures just become task failures
If the AI messes this up
Use this when the AI fakes progress or breaks the feature. It forces a real fix.
Call sites are untagged and unmeasurable. Add the class tag to the single model-client wrapper (create one if calls are scattered - that wrapper is also where routing will live).