EXPLAINER

Structured Outputs from LLMs

conceptfreshness: volatilesources not yet verified

Structured outputs make a model return data in a defined shape — usually JSON matching a schema — so software can use it without brittle parsing.

Short answer: Get machine-readable output by combining a schema, a model feature that enforces or encourages it (structured-output mode, function calling, constrained decoding), and validation in your code. Always validate; valid shape ≠ true content.

What is it?

Programs need data, not paragraphs. Structured outputs is the practice of getting a model to return JSON (or another fixed shape) your code can read directly — customer name, date, category — instead of free text you must parse.

You can ask in the prompt, use a built-in "structured output" feature, rely on function calling, or force the format with constrained decoding.

Also known as: structured output, JSON mode, JSON output from LLM, schema output, get JSON from GPT, typed output, output parsing, Pydantic output.

Technical definition

Mechanisms in rough order of strength: prompt instructions with examples (best-effort); "JSON mode" (valid JSON, schema not guaranteed, provider-dependent); schema-enforced structured-output features; function/tool calling with a parameter schema (Function Calling (Tool Calling) Explained); token-level Constrained Decoding (Guided Generation). Regardless of mechanism, applications should validate against JSON Schema or typed models and handle refusals, truncation and semantic errors. Feature availability and schema subsets are provider-specific.

Entity type: concept · Freshness class: volatile.

When should I use it?

Use whenever model output feeds code: extraction, classification, routing, tool arguments, UI rendering.

Implementation patterns

  • Define the schema once; reuse for prompt, enforcement and validation.
  • Include nullable/"unknown" fields and enums.
  • Parse → validate → retry with the error message on failure.
  • Keep a free-text justification field separate from the data fields.
  • Validate semantics (dates, ranges, foreign keys) after schema validation.

Technologies and frameworks

Commonly associated: JSON Schema, Pydantic, Zod, provider structured-output modes, Outlines, Instructor-style wrappers. Names are examples of what exists, not recommendations, and projects change quickly.

Practical guide

  1. Write the schema (names, types, enums, required).
  2. Pick the strongest mechanism your model/provider supports.
  3. Add "unknown" options.
  4. Validate in code, then business rules.
  5. Retry with the error once or twice, then fail safely.
  6. Log failures for evaluation.

Common failure modes

  • Invalid JSON from prompt-only approaches (extra prose, trailing commas, markdown fences).
  • Truncated JSON at max tokens.
  • Schema-valid but fabricated values.
  • Relying on JSON mode and assuming the schema is enforced.
  • Schemas too large or deeply nested for the provider's limits.

Troubleshooting

  • Extra text around JSON — Use a structured-output feature or constraint rather than prompt-only; strip fences defensively.
  • Missing required fields — Enforce a schema and mark required; retry with the validation error.
  • Fields hallucinated — Allow null/unknown; ground with source text; verify.

Contrasted with

Questions this page answers

  • How do I get structured output from an LLM?
  • What is JSON mode?
  • Why does my LLM return invalid JSON?
  • Should I use function calling or structured outputs?
  • How do I validate LLM JSON output?

Typical goals: get reliable JSON from a model; extract fields from text; choose a structured-output method.

Sources and verification

Status: not independently verified. This entry was authored on 2026-10-06 from general technical knowledge, without live checking of the sources below. Treat every source as a lead to confirm (identifier, title, authorship, current version) before publication or citation.

Freshness class: volatile — Products, versions, protocols, benchmarks, leaderboards, regulations and dates. Verify against the primary source before relying on specifics; re-review every 3 months.

Time-sensitive claims to verify before relying on them:

  • Which providers support schema-enforced output, supported schema subsets and limits.
  • Behaviour of "JSON mode" variants.

Canonical sources (unverified leads):

  • JSON Schema specification — json-schema.org
  • Provider documentation on structured outputs / JSON mode / function calling (identify current page) — locator to be identified
Learning path: Structured output and constrained decoding (step 2 of 6)
← Sampling and Decoding (Temperature, Top-p, Beam Search)JSON Schema →