Implementation:Predibase Lorax Response Type

Knowledge Sources	LoRAX
Domains	API_Design, Text_Generation
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for structured inference result representation provided by the Response, Details, and Token Pydantic models.

Description

The Response class wraps generated text with optional Details containing finish reason, token counts, and per-token Token objects. FinishReason is an enum with three values: Length, EndOfSequenceToken, and StopSequence. Each Token carries ID, text, logprob, special flag, and optional alternative tokens.

Usage

Returned by Client.generate() calls. Access response.generated_text for output and response.details for metadata when details=True was set in the request.

Code Reference

Source Location

• Repository: LoRAX
• File: clients/python/lorax/types.py
• Lines: 289-357

Signature

class Token(BaseModel):
    id: int
    text: str
    logprob: Optional[float]
    special: bool
    alternative_tokens: Optional[List[AlternativeToken]] = None
    skipped: bool

class FinishReason(str, Enum):
    Length = "length"
    EndOfSequenceToken = "eos_token"
    StopSequence = "stop_sequence"

class Details(BaseModel):
    finish_reason: FinishReason
    prompt_tokens: int
    generated_tokens: int
    skipped_tokens: int
    seed: Optional[int] = None
    prefill: List[InputToken]
    tokens: List[Token]
    best_of_sequences: Optional[List[BestOfSequence]] = None

class Response(BaseModel):
    generated_text: str
    details: Optional[Details] = None

Import

from lorax.types import Response, Details, Token, FinishReason

I/O Contract

Inputs

Name	Type	Required	Description
Server JSON response	Dict	Yes	Raw JSON from LoRAX /generate endpoint

Outputs

Name	Type	Description
Response.generated_text	str	The generated output text
Response.details	Optional[Details]	Token-level metadata (when details=True)
Details.finish_reason	FinishReason	Why generation stopped
Details.generated_tokens	int	Number of tokens generated

Usage Examples

Evaluating Merged Adapter Output

from lorax import Client
from lorax.types import MergedAdapters

client = Client("http://localhost:3000")

response = client.generate(
    "Classify this review as positive or negative: 'Great product!'",
    merged_adapters=MergedAdapters(
        ids=["sentiment-v1", "sentiment-v2"],
        weights=[0.5, 0.5],
        merge_strategy="linear",
        density=1.0,
    ),
    max_new_tokens=10,
    details=True,
)

print(f"Output: {response.generated_text}")
print(f"Reason: {response.details.finish_reason}")
print(f"Tokens: {response.details.generated_tokens}")

# Check per-token confidence
for token in response.details.tokens:
    print(f"  {token.text}: logprob={token.logprob:.3f}")

Related Pages

Implements Principle

• Principle:Predibase_Lorax_Inference_Result_Evaluation