Principle:Guardrails ai Guardrails Structured Output Strategy
| Knowledge Sources | |
|---|---|
| Domains | Structured_Output, LLM_Integration |
| Last Updated | 2026-02-14 00:00 GMT |
Overview
A strategy selection principle for choosing the optimal method to extract structured JSON data from LLMs based on provider capabilities and output requirements.
Description
Structured Output Strategy addresses the problem of extracting reliably formatted JSON from LLMs, which natively produce free-form text. Guardrails supports multiple strategies, each with different trade-offs:
- Function Calling: Uses the OpenAI-compatible tools parameter to define the output schema as a function signature, guiding the LLM to produce structured JSON as a "function call"
- JSON Mode (response_format): Uses the response_format parameter with a JSON schema to request strict JSON output
- Constrained Decoding (Jsonformer): Intercepts the generation process to force valid JSON at the token level, only available for locally-hosted HuggingFace models
- Prompt Suffixing: Appends JSON schema instructions to the prompt text, relying on the LLM to follow formatting instructions
Usage
Choose function calling for OpenAI/Anthropic models with tool support. Use JSON mode (response_format) for models supporting strict JSON schemas. Use constrained decoding for local HuggingFace models where format compliance is critical. Fall back to prompt suffixing for models without native structured output support.
Theoretical Basis
Each strategy operates at a different level of the generation pipeline:
| Strategy | Level | Guarantee | Provider Support |
|---|---|---|---|
| Function Calling | API parameter | High (schema-guided) | OpenAI, Anthropic, most providers |
| JSON Mode | API parameter | High (strict mode) | OpenAI, some providers |
| Constrained Decoding | Token generation | Absolute (forced at decode) | Local HuggingFace only |
| Prompt Suffix | Prompt text | Low (LLM compliance) | All providers |