Implementation:Predibase Lorax Response Format Type

Knowledge Sources	LoRAX
Domains	Structured_Output, API_Design
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for specifying JSON output format constraints provided by the ResponseFormat and ResponseFormatType types.

Description

The ResponseFormat Pydantic model wraps a ResponseFormatType enum (currently json_object) with an optional schema_spec field containing the JSON Schema dict. The schema field is aliased in serialized JSON to match the OpenAI API format. On the Rust server side, a corresponding ResponseFormat struct with ResponseFormatType enum handles deserialization.

Usage

Create a ResponseFormat instance with your JSON Schema and pass it as response_format to Client.generate() or the OpenAI chat completions API.

Code Reference

Source Location

• Repository: LoRAX
• File: clients/python/lorax/types.py (Lines: 61-71)
• File: router/src/lib.rs (Lines: 525-553)

Signature

class ResponseFormatType(str, Enum):
    json_object = "json_object"

class ResponseFormat(BaseModel):
    model_config = ConfigDict(use_enum_values=True)

    type: ResponseFormatType
    schema_spec: Optional[Union[Dict[str, Any], OrderedDict]] = Field(
        None, alias="schema"
    )

// Rust-side (router/src/lib.rs)
pub(crate) enum ResponseFormatType {
    Text,
    JsonObject,
    JsonSchema,
}

pub(crate) struct ResponseFormat {
    pub r#type: ResponseFormatType,
    pub schema: Option<serde_json::Value>,
}

Import

from lorax.types import ResponseFormat, ResponseFormatType

I/O Contract

Inputs

Name	Type	Required	Description
type	ResponseFormatType	Yes	Must be "json_object"
schema_spec	Optional[Dict/OrderedDict]	No	JSON Schema dict (aliased as "schema" in JSON)

Outputs

Name	Type	Description
response_format	ResponseFormat	Validated format specification for constrained decoding

Usage Examples

With Pydantic Schema

from pydantic import BaseModel
from lorax import Client
from lorax.types import ResponseFormat

class ExtractedEntity(BaseModel):
    name: str
    entity_type: str
    confidence: float

client = Client("http://localhost:3000")
response = client.generate(
    "Extract the entity from: 'Apple released the iPhone 15'",
    response_format=ResponseFormat(
        type="json_object",
        schema=ExtractedEntity.model_json_schema(),
    ),
    adapter_id="my-extraction-adapter",
    max_new_tokens=100,
)
import json
entity = json.loads(response.generated_text)
# {"name": "Apple", "entity_type": "company", "confidence": 0.95}

With Manual Schema

response = client.generate(
    "What is 2+2?",
    response_format=ResponseFormat(
        type="json_object",
        schema={
            "type": "object",
            "properties": {
                "answer": {"type": "integer"},
                "explanation": {"type": "string"},
            },
            "required": ["answer"],
        },
    ),
    max_new_tokens=50,
)

Related Pages

Implements Principle

• Principle:Predibase_Lorax_JSON_Schema_Definition