Principle:Predibase Lorax Inference Request Construction

Knowledge Sources	LoRAX API Reference Nucleus Sampling
Domains	API_Design, Text_Generation
Last Updated	2026-02-08 02:00 GMT

Overview

A request specification pattern that combines prompt text, adapter selection, and generation parameters into a validated inference request with mutual exclusivity constraints.

Description

Inference Request Construction defines the contract between client and server for text generation. The key design decisions are:

• Adapter Selection: Each request can specify either a single adapter_id or merged_adapters (but not both), enabling per-request LoRA customization
• Generation Parameters: Standard sampling controls (temperature, top_k, top_p, repetition_penalty) with validation constraints
• Streaming: Requests can opt into Server-Sent Events streaming for token-by-token output
• Schema Constraints: Optional response_format for structured JSON output

Usage

Use this principle when constructing any inference request to a LoRAX server. The Parameters class encapsulates all generation settings and adapter configuration.

Theoretical Basis

Request validation follows constraint propagation:

Pseudo-code:

# Validation rules
assert adapter_id is None or merged_adapters is None  # mutual exclusion
assert temperature >= 0 if temperature is not None
assert 0 < top_p < 1 if top_p is not None
assert best_of > 1 implies do_sample = True
assert best_of > 1 implies stream = False

Related Pages

Implemented By

• Implementation:Predibase_Lorax_Parameters_Request_Types