Structured Outputs

Definition

Structured outputs is a large language model feature that constrains a model's response to strictly conform to a predefined schema, such as JSON with specific fields and types, guaranteeing reliably parseable output rather than freeform text.

Overview

By default, an LLM generates free-form text, and while it can be instructed to "respond in JSON," nothing prevents it from occasionally producing malformed JSON, extra commentary, or fields that don't match an expected schema — a real problem for any downstream code that needs to parse and act on the model's output programmatically. Structured outputs solve this by constraining the model's generation process itself, typically through constrained decoding: at each generation step, the model is only allowed to produce tokens consistent with completing a valid instance of the specified schema, mathematically guaranteeing schema-conformant output rather than merely making it more likely through prompting. Developers specify the desired schema — commonly using JSON Schema — describing required fields, types, enums, and nesting, and the model provider's API enforces that the output strictly matches it. This is distinct from simply asking a model nicely to format its response a certain way, since prompting alone can still fail on edge cases, adversarial inputs, or longer generations, while true structured output enforcement removes that failure mode at the decoding level. Structured outputs are closely related to, and often combined with, function calling — the arguments a model provides when calling a function are themselves a form of structured output constrained to that function's parameter schema. Beyond function calling, structured outputs are widely used any time an LLM's response needs to feed directly into other software: extracting structured data from documents, classifying text into a fixed set of categories, or generating configuration files, where any parsing failure would break an automated pipeline. Anthropic, OpenAI, and other major providers offer structured output support in their APIs, though the specific enforcement mechanisms and guarantees vary across providers.

Key Concepts

• Constrains model output to strictly match a predefined schema, commonly JSON Schema
• Uses constrained decoding to mathematically guarantee schema conformance
• Removes parsing failures caused by malformed or off-schema model output
• Distinct from simply prompting a model to "respond in JSON"
• Closely related to and often combined with function calling
• Supports required fields, types, enums, and nested object structures
• Enables reliable integration of LLM output into automated software pipelines
• Supported across major provider APIs, including Anthropic and OpenAI

Use Cases

Frequently Asked Questions

Related in AI Fundamentals