Structured Outputs

A structured output request gives the model an explicit target shape. With OpenAI, that commonly means response_format with type json_schema, a named schema, and strict:true. With Anthropic, the most reliable pattern is tool-based structuring: define a tool input schema, instruct the model to use that tool, then read the tool input as the structured object. With Gemini, generation config can set a JSON MIME type and responseSchema so the model emits data matching the declared schema.

The practical effect is that downstream code no longer scrapes natural language. It receives an object such as a classification, extraction, route decision, or rubric score. The schema becomes an API boundary between the LLM and the rest of the system. That boundary should be reviewed, versioned, tested, and monitored like any other interface.

Plain JSON mode is useful but weaker. It tells the model to produce valid JSON, so the output is usually parseable by a JSON parser. It does not guarantee that priority is one of low, medium, or high, that required fields are present, that extra fields are absent, or that nested objects match your domain model.

Structured outputs add schema-level constraints. A strict schema can require fields, disallow unknown properties with additionalProperties:false, constrain strings with enums, and express limited unions with anyOf. If software depends on exact keys and values, structured outputs are the correct default; JSON mode is mainly for loose human-assisted workflows or quick prototypes.

Under the hood, providers compile the supported schema into a grammar, finite-state machine, or similar constraint representation. At each decoding step, the sampler only allows tokens that can still lead to a valid object. Tokens that would break the schema are masked out before sampling.

That is why schema-constrained decoding can guarantee parseability for normal completions. If the next legal character must be a quote, a comma, a closing brace, or one of a small set of enum values, the model cannot freely emit arbitrary prose. The model still chooses among allowed continuations, so the content can be wrong or low quality, but the container is valid.

OpenAI exposes strict structured output through response_format json_schema, including a schema name and strict:true. Modern SDKs may also provide parse helpers that return a typed parsed object when the response is successful.

Anthropic often uses tools for structure. A tool definition includes an input_schema, and the assistant emits a tool_use block with JSON input. You may force a specific tool choice when the only acceptable successful output is that structured payload. Gemini supports responseSchema with response MIME type application/json, so the response is generated as JSON matching the schema shape supported by the API. The names differ, but the application pattern is the same: declare the shape, parse only the normal structured channel, and treat refusals or safety blocks separately.

Provider-supported schemas are intentionally narrower than full JSON Schema. Common constraints include object types, properties, required arrays, additionalProperties:false, enums, arrays, primitive types, and limited anyOf unions. Some providers require every property to be listed as required, with optionality represented by an explicit nullable union such as string or null.

Advanced JSON Schema features may be rejected or ignored: arbitrary patternProperties, complex conditional validation, deeply recursive definitions, unbounded recursion, very deep nesting, huge enums, or root-level unions. Treat the provider schema subset as a product constraint. Keep schemas small, explicit, and close to what the model must decide.

A safety refusal should not be forced into your business schema. If the user asks for disallowed content, the provider can surface a refusal or safety block through a separate field, message part, stop reason, or candidate status. In that case, the application should not attempt to parse the normal schema payload.

This separation is important for correctness and security. A normal parsed object means the model accepted the task and produced schema-shaped data. A refusal means the task was not completed as structured data. Your code path should branch early: handle refusal UX, audit the event, and avoid feeding a fake parsed object into downstream automation.

Guided decoding can guarantee that the response is parseable and schema-shaped, but it cannot guarantee that the extracted date is correct, that the classification is fair, or that the answer is grounded in evidence. The model still predicts content based on prompt, context, and learned behavior.

This distinction matters in interviews. Structured outputs solve interface reliability. They do not solve factuality, authorization, prompt injection, or business-rule enforcement. You still need grounding, retrieval quality, server-side checks, and evaluation.

If a downstream workflow has known states, use enums. A priority field with low, medium, high, and urgent is easier to validate, measure, and route than a free-text priority_reason field that the model can phrase in many ways.

Enums also reduce decoding ambiguity. The model chooses from a small set of allowed continuations, which improves repeatability and reduces accidental synonyms. Free text should be reserved for human-readable explanations, evidence snippets, or fields where open-ended language is truly required.

Many strict schema implementations work best when every property is required. That can feel odd for optional fields, but it is usually better to require the field and allow null than to let the key disappear.

Explicit null has product value. It forces the prompt and schema to define what missing evidence means. For example, renewal_date can be a string or null, and a separate missing_fields array can explain what evidence was absent. Downstream code no longer guesses whether a missing key means unknown, unsupported, or model error.

A deeply nested schema may be valid for software but hard for the model to fill consistently. Every nested level increases the number of braces, arrays, required fields, and decisions the decoder must satisfy. It can also make provider limits more likely.

Prefer flat or moderately nested objects for LLM outputs. If the business object is complex, split the task into stages: extract core facts first, validate them, then call a second schema for enrichment or planning. Smaller schemas are easier to test and easier to repair.

Strict structured outputs reduce malformed JSON, but repair remains useful for non-strict fallbacks, provider outages, domain validation failures, and business constraints that are outside the schema subset. For example, a schema can require a date string, but your validator may reject dates before account creation.

A good repair loop is bounded and observable. Send the validation error, the original task, and the invalid object to a repair prompt with the same schema. Retry once or twice, then fall back to a human review queue or safe default. Infinite repair loops turn model uncertainty into latency and cost.

Schema changes are API changes. Adding an enum value, renaming a field, or changing nullable behavior can break parsers, dashboards, evaluation cases, and downstream workflow rules. Record schema_version with each request and tie it to the prompt and model version.

Track whether each request ended as parsed, refused, blocked, validation_failed, repaired, or fallback. This makes quality regressions visible. A rising validation failure rate usually indicates prompt drift, schema mismatch, a provider behavior change, or new user inputs not represented in tests.

Do not ask the model to enforce permissions, billing policy, or irreversible side-effect rules just because the output is structured. Use application code for authorization, quotas, idempotency, data access, and final tool execution decisions.

Schemas consume request tokens and constrained decoding can add latency depending on provider implementation, schema size, and enum complexity. The tradeoff is often worth it for automation, but high-volume paths should measure cost, latency, and failure rate against simpler classifiers or deterministic rules.

This Python example defines a schema for ticket triage, sends it as an OpenAI strict json_schema response_format, checks for refusal first, then parses the normal response. It uses string concatenation and ordinary JSON parsing so the control flow is clear.

Strict schemas protect the interface, while application validation protects product rules. This example rejects an object that is schema-shaped but violates a domain rule, then prepares a bounded repair prompt.

Tool-based structuring represents the desired payload as tool arguments. In an Anthropic-style flow, the application reads the tool_use input rather than parsing assistant prose.