Principle:Predibase Lorax OpenAI Compatible Client
| Knowledge Sources | |
|---|---|
| Domains | API_Compatibility, Client_SDK |
| Last Updated | 2026-02-08 02:00 GMT |
Overview
An API compatibility pattern that allows standard OpenAI SDK clients to connect to a LoRAX server by pointing the base URL to the LoRAX endpoint.
Description
OpenAI Compatible Client addresses the adoption barrier of proprietary inference servers. By implementing the OpenAI Chat Completions API specification, LoRAX allows existing applications built on the OpenAI SDK to switch to self-hosted inference without code changes. The key mapping is:
- base_url points to the LoRAX server (e.g., http://localhost:3000/v1)
- api_key can be any string (no auth by default)
- model parameter is repurposed as the LoRA adapter ID
This enables a drop-in replacement pattern for OpenAI API consumers.
Usage
Use this principle when you want to interact with LoRAX using the standard OpenAI Python SDK. This is the recommended approach for chat-style interactions and for applications that need to support both OpenAI and self-hosted backends.
Theoretical Basis
The compatibility is achieved through API surface mimicry:
Pseudo-code:
# OpenAI SDK → LoRAX mapping
openai_client = OpenAI(
base_url="http://lorax-server:3000/v1", # Redirect to LoRAX
api_key="not-needed", # Placeholder
)
# The model parameter becomes the adapter_id
response = openai_client.chat.completions.create(
model="my-org/my-lora-adapter", # → adapter_id
messages=[...], # Standard chat format
)