Principle:Predibase Lorax Inference Result Evaluation

Knowledge Sources	LoRAX Python Client
Domains	API_Design, Text_Generation
Last Updated	2026-02-08 02:00 GMT

Overview

A structured response evaluation pattern that provides generated text alongside token-level metadata including finish reasons, logprobs, and generation statistics.

Description

Inference Result Evaluation provides a structured way to assess the quality and completeness of model outputs. Key information includes:

• Finish Reason: Why generation stopped (length = hit max_new_tokens, eos_token = model chose to stop, stop_sequence = matched a stop string)
• Token Details: Per-token IDs, text, logprobs, and special token flags
• Generation Statistics: Prompt tokens, generated tokens, skipped tokens (from speculative decoding)
• Alternative Tokens: Top-k alternative token probabilities (when requested)

This enables users to programmatically assess whether output was truncated, evaluate model confidence, and debug generation behavior.

Usage

Use when evaluating model output quality, especially during adapter comparison for merged adapter experiments. Pass details=True to get full token-level information.

Theoretical Basis

Pseudo-code:

# Response evaluation
response = client.generate(prompt, details=True)
if response.details.finish_reason == "length":
    # Output was truncated, consider increasing max_new_tokens
    pass
elif response.details.finish_reason == "eos_token":
    # Model naturally completed generation
    pass
# Check confidence via average logprob
avg_logprob = mean([t.logprob for t in response.details.tokens])

Related Pages

Implemented By

• Implementation:Predibase_Lorax_Response_Type