Tool Descriptions That Steer the Model
Treat tool descriptions as prompt engineering: when-to-use guidance, negative cases, and worked examples that fix tool-choice errors.
The route
3 steps to Done
- 01
Collect misroutes and map overlaps
Fix the confusions you actually have, not hypothetical ones.
Preview prompt + verify gate ▾ Hide ▴
Gather the evidence. From recent transcripts, collect every tool misroute: cases where the model used tool A when tool B was correct (shell grep instead of the search tool, full-file write instead of targeted edit, web search for local questions). Then map the overlap structure of my toolset: which tools can plausibly handle the same request? For each overlapping pair, write the one-sentence boundary rule (e.g. 'edit for changes within a file; write only for new files or full regenerations'). Deliver the misroute list and the boundary rules - these drive every rewrite.
- ✓ Misroutes cite real transcript cases
- ✓ All overlapping pairs identified
- ✓ Each boundary is one testable sentence
- 02
Rewrite descriptions to steer
Purpose, use-when, avoid-when, example - in that order, tightly.
Preview prompt + verify gate ▾ Hide ▴
Rewrite every tool description with the steering template: (1) PURPOSE - one sentence, what it accomplishes; (2) USE WHEN - the 2-3 concrete situations it is the right choice; (3) DO NOT USE FOR - the adjacent situations, each naming the correct alternative tool (mirror this in the other tool's description); (4) EXAMPLE - one compact input/outcome pair for non-trivial tools. Apply the boundary rules from the previous step verbatim in the relevant pairs. Budget: ~80-120 tokens per description. Rewrite all of them now and show the before/after for the three most misrouted tools.
- ✓ All four template parts present
- ✓ Alternatives named in every negative case
- ✓ Token budget held
- 03
Verify with a routing test
Tool choice is testable - so test it.
Preview prompt + verify gate ▾ Hide ▴
Build and run the routing test. Write 12 scenarios as realistic user requests, each with one known-correct tool (include every misroute case from the inventory, each overlapping boundary from both sides, and 2 trap scenarios where the superficially-obvious tool is wrong). For each scenario, run the agent and record the first tool it reaches for. Score routing accuracy before (old descriptions, if measurable) and after the rewrite. Requirement: 12/12 with the new descriptions. For any failure, diagnose whether the gap is the description (fix the wording) or a genuinely ambiguous toolset (split or merge tools), then re-run.
- ✓ Scenarios cover misroutes, boundaries, and traps
- ✓ First-tool-choice is the scored signal
- ✓ Failures diagnosed as wording vs toolset
Research-backed