Deferred Tool Loading

The route

4 steps to Done

1. 01

   Measure usage and pick the core

   The data decides what stays eager.

   Preview prompt + verify gate ▾ Hide ▴

   Analyze tool usage to define the core set. From transcript logs, compute per-tool: call frequency across sessions, task-type coverage (which task categories touch it), and first-call timing (tools needed in early turns favor eager loading). Also measure the token cost of each tool's definition. Core-set criteria: high frequency OR early-turn necessity OR safety-critical (permission gates should not be deferred). Everything else defers. Deliver: the ranked usage table, the core list (aim for the smallest set covering ~90% of calls), and the projected context savings from deferring the rest.

   • ✓ Frequency and timing measured from real logs
   • ✓ Safety-critical tools kept eager
   • ✓ Savings projection computed
2. 02

   Build the catalog and discovery

   The model can only load what it can find.

   Preview prompt + verify gate ▾ Hide ▴

   Implement discovery. Build the deferred-tool catalog: one line per tool - name plus a purpose phrase written like a steering description in miniature ('code_metrics: complexity and duplication reports for source files'). Include the catalog in context under a header explaining: 'additional tools exist; call load_tools([names]) to activate the ones your task needs'. Group the catalog by capability area for scannability. Budget the whole block tightly (it must be far cheaper than the definitions it replaces). Add a search_tools(query) function for large catalogs. Verify the model reads it: give a task needing a deferred tool and watch for the load call.

   • ✓ One steering line per deferred tool
   • ✓ Catalog cost is a small fraction of definitions
   • ✓ Model issues load_tools unprompted in the test
3. 03

   Implement dynamic registration

   Mid-session loading that keeps the loop and cache sane.

   Preview prompt + verify gate ▾ Hide ▴

   Implement load_tools(names). On call: validate names against the catalog; register the requested tools' full definitions into the session's active toolset; return a confirmation listing the loaded tools with their descriptions; keep them active for the session's remainder. Cache-awareness: append newly-loaded definitions in a stable position per your provider's tool-definition handling (accept the one-time cache reset per load and log it; batch-load when the model requests several). Handle the errors: unknown names get nearest-match suggestions; already-loaded is an idempotent no-op. Verify a loaded tool round-trips: load, call, correct result.

   • ✓ Loaded tools callable immediately
   • ✓ Unknown names get suggestions
   • ✓ Cache resets logged and batched
4. 04

   Measure savings and guard capability

   Prove the win and prove nothing broke.

   Preview prompt + verify gate ▾ Hide ▴

   Run the evaluation. SAVINGS: measure tool-definition tokens per session before vs after deferral across 10 representative tasks; report average and range. CAPABILITY: run a regression suite of tasks that collectively require several deferred tools; success requires the model discovering, loading, and using each without human help; compare completion rates against the all-eager baseline. DISCOVERY FAILURES: for any task where the model struggled without the right tool, diagnose (catalog wording? missing search? tool should be core?) and fix. Institute a monthly usage re-check - core membership follows the data as usage evolves.

   • ✓ 10-task savings report produced
   • ✓ Regression matches all-eager completion
   • ✓ Discovery failures diagnosed and fixed

Research-backed

Sources behind this flow