Structured Output from Tools

The route

4 steps to Done

1. 01

   Map the boundaries and define types

   Find every place prose crosses into code.

   Preview prompt + verify gate ▾ Hide ▴

   Inventory every model-to-system boundary in my pipeline: tool-call arguments, extraction outputs, classification results, generation outputs consumed by code, and inter-agent messages. For each, define the output type as a schema: precise field types, enums for closed sets, optionality that reflects reality, and field descriptions (they double as generation guidance). Flag the boundaries currently parsed by regex or string-splitting - these are the incident factories and get migrated first. Deliver the boundary inventory with schemas drafted for the top 5.

   • ✓ All boundaries listed with current parsing method
   • ✓ Regex boundaries flagged for priority
   • ✓ Schemas drafted with enums and descriptions
2. 02

   Enforce schemas at generation time

   Prevention beats parsing: make the model emit the type.

   Preview prompt + verify gate ▾ Hide ▴

   Wire schema enforcement into generation. Where the provider supports native structured output (JSON mode / schema-constrained decoding / tool-call shaped output), use it with the defined schema - this eliminates most malformation at the source. Where not, instruct with the schema in the prompt ('Return ONLY JSON matching this schema') and validate after. Either way, ALWAYS validate the parsed result against the schema at runtime (native modes reduce but do not eliminate violations - ranges and cross-field rules still need checking). Migrate the top-5 boundaries now and delete their regex parsers. Run existing flows to confirm outputs still satisfy consumers.

   • ✓ Native modes used where available
   • ✓ Runtime validation on every boundary regardless
   • ✓ Regex parsers deleted, flows green
3. 03

   Build the repair loop and fallbacks

   Plan for malformation like it is weather, not scandal.

   Preview prompt + verify gate ▾ Hide ▴

   Implement graceful degradation. REPAIR: on validation failure, send one repair request containing the invalid output and the specific validation errors ('field X: expected one of [a,b], got c') with the instruction to return corrected JSON only; validate again; hard cap of one repair round (two for high-value outputs) - repair loops must be bounded. FALLBACK: per output type, define the safe floor - a typed minimal object marked as degraded (e.g. classification falls back to 'unknown' with confidence 0), or a queued human-review path for outputs too important to floor. Fallbacks always log and alert; they never silently impersonate real data. Test the ladder by injecting malformed outputs: common breakage repairs; garbage falls back visibly.

   • ✓ Repair prompt carries the exact validation errors
   • ✓ Rounds hard-capped
   • ✓ Fallbacks typed, marked degraded, and alerting
4. 04

   Add telemetry and hold the line

   Validation rates are pipeline health - watch them.

   Preview prompt + verify gate ▾ Hide ▴

   Operationalize. Emit telemetry per boundary: first-pass validation rate, repair-success rate, fallback rate, per model and prompt version - a dashboard or periodic report both work. Set alert thresholds (e.g. fallback rate above 2% pages someone; first-pass below 90% opens an investigation). Add CI contract tests: golden examples per boundary validating against the schema, so schema changes and prompt changes get checked together. Finish with a chaos check: temporarily point one boundary at a weaker model and confirm the ladder absorbs the quality drop visibly (repair rate rises, nothing crashes) - that is the system working.

   • ✓ Rates tracked per boundary and version
   • ✓ Alert thresholds set
   • ✓ Chaos check absorbed visibly

Research-backed

Sources behind this flow