Design Tool Schemas for Agents

The route

4 steps to Done

1. 01

   Audit schemas for looseness

   Every loose type is an invitation the model will accept.

   Preview prompt + verify gate ▾ Hide ▴

   Audit all my tool schemas for constraint weakness. For each parameter record: current type, whether its real domain is narrower (a string that is actually one of 4 modes, a number that must be 1-100), whether it is truly required, whether its description would let a newcomer produce a valid value, and misuse evidence from transcripts (wrong shapes the model has actually sent). Rank the tools by misuse frequency. Deliver the audit table with a proposed tightened type for every weak parameter.

   • ✓ Every parameter assessed against its real domain
   • ✓ Transcript misuse evidence attached
   • ✓ Tightening proposals concrete
2. 02

   Tighten the schemas

   Make illegal states unrepresentable in the call.

   Preview prompt + verify gate ▾ Hide ▴

   Apply the tightening. Rules: strings with finite options become enums; numbers get minimum/maximum; identifiers get pattern constraints where formats exist (paths, slugs); arrays get minItems/maxItems and item types; every field gets a description with purpose + format + example; defaults documented in the description AND implemented in the handler; additionalProperties set to false everywhere. For genuinely free-text fields (a commit message, a search query), keep them but add length bounds and a description stating exactly what good input looks like. Regenerate the tool definitions and re-run existing happy-path tests to confirm nothing legitimate broke.

   • ✓ Enums, ranges, and patterns applied
   • ✓ additionalProperties false on all tools
   • ✓ Happy-path tests pass post-tightening
3. 03

   Make validation errors teach

   The error message is the model's documentation at the moment it matters most.

   Preview prompt + verify gate ▾ Hide ▴

   Rebuild validation error responses as corrective teaching. Format: which field failed, what was received, what is allowed (for enums: list the values; for ranges: state them; for patterns: show a valid example), plus a complete corrected example call when cheap to construct. Reject unknown parameters by name ('unknown parameter: file - did you mean path?') with nearest-match suggestions. Ensure errors return to the model as structured tool results, not exceptions. Test by replaying the misuse suite and reading each error as if you were the model: could you fix the call from the message alone?

   • ✓ Errors name field, received, and allowed
   • ✓ Unknown params get nearest-match hints
   • ✓ Misuse replay shows fixable messages
4. 04

   Lock it in with a misuse suite

   Schemas regress silently unless tests hold the line.

   Preview prompt + verify gate ▾ Hide ▴

   Build the permanent misuse suite. Collect at least 10 bad calls: real transcript misuses plus constructed edge cases (out-of-range numbers, invalid enum values, extra parameters, missing required fields, wrong types, oversized arrays). Assert each is rejected AND the error message contains the allowed values or a corrected hint. Add 5 boundary calls that must PASS (values exactly at limits). Wire the suite into the test run so schema changes cannot land without it green. Finish with a live check: run the agent on 3 tool-heavy tasks and compare misuse frequency against the pre-tightening baseline.

   • ✓ 10 rejects and 5 boundary passes asserted
   • ✓ Error content asserted, not just status
   • ✓ Live misuse compared to baseline

Research-backed

Sources behind this flow