Structured Outputs

2 min · concept

Structured outputs are techniques for making a model return data in a specific machine-readable format — JSON, XML, or any schema you define — rather than prose. Getting reliable structured output is one of the first practical problems in production LLM engineering.

Two Approaches

JSON mode — instruct the model to emit valid JSON via the system prompt. The API guarantees syntactically valid JSON but not schema compliance. You must still validate against your schema. Simpler to set up; fragile under distribution shift.

Tool use / function calling — define the desired schema as a tool, force the model to call it with tool_choice. The model generates a tool_use block; its input field is your structured output. More reliable than JSON mode because the model is optimized to produce schema-valid tool calls. See tool-use for the full mechanics.

import anthropic

client = anthropic.Anthropic()

# Tool use as structured output extraction
tools = [{
    "name": "extract_event",
    "description": "Extract event details from text.",
    "input_schema": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "date": {"type": "string", "description": "ISO 8601"},
            "location": {"type": "string"}
        },
        "required": ["name", "date"]
    }
}]

response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "extract_event"},
    messages=[{"role": "user", "content": "The product launch is June 12 in Austin."}]
)

result = response.content[0].input  # {"name": "product launch", "date": "2025-06-12", "location": "Austin"}

Schema Definition

Define schemas with JSON Schema directly, or use a library that serializes to it:

Python — Pydantic model: model.model_json_schema() → JSON Schema
TypeScript — Zod: z.object({...}) + a library like zod-to-json-schema

Keep schemas flat where possible. Deeply nested objects increase error rates. Optional fields with defaults outperform missing required fields.

Constrained Decoding

Some inference engines (llama.cpp, vLLM, Outlines) use constrained decoding to guarantee schema compliance at the token generation level. Rather than sampling freely and hoping for valid JSON, the decoder masks tokens that would produce invalid output. Zero invalid outputs, but the approach requires local inference or a provider that supports it. OpenAI's Structured Outputs feature uses constrained decoding on their infrastructure.

Validation Strategy

Never trust model output blindly, even with constrained decoding in the loop.

Parse first: json.loads() / model.model_validate()
Validate against schema second
Handle failures explicitly: retry, fallback, or surface the error — do not silently swallow a parse failure and continue downstream

Retry budgets matter. One retry on parse failure recovers most cases. Three retries without model state mutation rarely helps.

Common Failure Modes

Model adds prose before or after the JSON block — fixed by strict tool calling
Enum values drift from the schema — use enum constraints in the schema
Nested optional fields omitted — set defaults rather than marking fields optional
Large schemas degrade instruction following — keep schemas under 20 fields when possible

tool-use