Tool Result Size Management

The route

4 steps to Done

1. 01

   Budget every tool's output

   The cap comes first; elegance comes after.

   Preview prompt + verify gate ▾ Hide ▴

   Implement output budgeting. Add a global default output budget (e.g. ~2000 tokens) with per-tool overrides where justified (file viewers higher, quick checks lower); enforce it in the tool execution wrapper so no individual tool can forget; measure output size before returning and route over-budget results into the truncation pipeline (next step). Add logging: every over-budget event records tool, actual size, and budget. Run a survey of recent transcripts to set data-informed budgets: what does each tool typically produce, and what did the model actually use?

   • ✓ Wrapper-level enforcement covers all tools
   • ✓ Budgets informed by transcript data
   • ✓ Over-budget events logged
2. 02

   Truncate to preserve signal

   Where the information lives decides how to cut.

   Preview prompt + verify gate ▾ Hide ▴

   Implement signal-preserving truncation. Default strategy: head+tail - keep the first H and last T tokens with a marker in between stating exactly what was elided ('[... 45,120 characters omitted - lines 210 to 3,847 ...]') and how to see more ('use read_output(offset) or narrow the command'). Rationale from practice: command errors and summaries cluster at the ends. Content-aware refinements: for failed commands, bias the split toward the tail where errors live; for structured formats (JSON), truncate at element boundaries with a count of omitted items rather than mid-token. Test on: a megabyte build log with the error at the end, a huge JSON array, and a giant file read - the model must see the signal in each.

   • ✓ Elision markers state size and retrieval path
   • ✓ Failed-command truncation biases to tail
   • ✓ Structured data cut at element boundaries
3. 03

   Paginate lists and reference artifacts

   Big results become navigable, not smaller - and huge ones become pointers.

   Preview prompt + verify gate ▾ Hide ▴

   Implement the two escapes from truncation. PAGINATION for list-like results: search hits, directory listings, and log queries return page 1 (budget-sized) plus total count and a next_page token; the tool accepts page parameters; ordering is stable across pages. REFERENCES for oversized artifacts: when a result exceeds a reference threshold (e.g. a generated report, a large diff), store it to a session artifact store, return {summary (produced by a quick summarization pass or structural digest), artifact_id, size, fetch instructions}; implement fetch_artifact(artifact_id, range?) for windowed retrieval. Test: a thousand-hit search navigated across pages, and a huge diff consumed via summary-then-targeted-fetch.

   • ✓ Stable ordering across pages
   • ✓ References carry a useful summary
   • ✓ Windowed fetch retrieves the right region
4. 04

   Regression-test the bounds

   Context bombs come back unless tests hold the perimeter.

   Preview prompt + verify gate ▾ Hide ▴

   Lock it in. Build the bounds suite: for every tool, a worst-case input (huge file, explosive command, thousand-hit query) asserting the result stays within budget AND the signal survives (error text present, marker present, fetch path valid). Add a context-level assertion in the loop: no single turn's tool results may exceed a global fraction of the window - log and alert if approached. Run the full agent on 3 previously-problematic heavy tasks and compare context usage curves against the pre-management baseline. Keep the suite in CI so budget regressions cannot land quietly.

   • ✓ Worst-case inputs per tool covered
   • ✓ Signal-survival asserted, not just size
   • ✓ Context curves improved on heavy tasks

Research-backed

Sources behind this flow