Step 1 of 4
Budget every tool's output
The cap comes first; elegance comes after.
Implement output budgeting. Add a global default output budget (e.g. ~2000 tokens) with per-tool overrides where justified (file viewers higher, quick checks lower); enforce it in the tool execution wrapper so no individual tool can forget; measure output size before returning and route over-budget results into the truncation pipeline (next step). Add logging: every over-budget event records tool, actual size, and budget. Run a survey of recent transcripts to set data-informed budgets: what does each tool typically produce, and what did the model actually use?
Expected after this step
Enforced budgets at the wrapper level with data-informed values.
Should not happen
- ✕Naive head-only truncation cutting off the error at the tail
- ✕Silent truncation the model does not know happened, breeding hallucinated completions
- ✕No pagination, so 'list files' becomes a context bomb
- ✕Dumping artifacts into context that should have been stored and referenced
Verify before continuing
Do not move on until every check is true. The complete button stays locked until then.
Do not continue if…
- !Naive head-only truncation cutting off the error at the tail
- !Silent truncation the model does not know happened, breeding hallucinated completions
- !No pagination, so 'list files' becomes a context bomb
- !Dumping artifacts into context that should have been stored and referenced
If the AI messes this up
Use this when the AI fakes progress or breaks the feature. It forces a real fix.
Individual tools bypass the budget. Enforcement must live in the single execution wrapper every call flows through - if tools return directly to the loop anywhere, close that path.