Implementation:Predibase Lorax OpenAI Client Configuration

Knowledge Sources	LoRAX OpenAI Python SDK
Domains	API_Compatibility, Client_SDK
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for configuring an OpenAI SDK client to connect to a LoRAX server, provided by the OpenAI Python SDK with base_url override.

Description

The OpenAI Python SDK's OpenAI client class accepts a base_url parameter that redirects all API calls to a custom endpoint. When pointed at a LoRAX server's /v1 path, the client seamlessly communicates with LoRAX's OpenAI-compatible chat completions endpoint. The server registers the route at /v1/chat/completions in router/src/server.rs.

Usage

Use this as the entry point for OpenAI-compatible chat interactions with LoRAX. Initialize the OpenAI client with the LoRAX server URL and use standard chat completion methods.

Code Reference

Source Location

• Repository: LoRAX
• File: router/src/server.rs
• Lines: 1353-1386 (route registration), 1633 (chat completions route)

Signature

# Client-side (external OpenAI SDK)
from openai import OpenAI

client = OpenAI(
    base_url: str,  # LoRAX server URL with /v1 path
    api_key: str,   # Any string (no auth by default)
)

// Server-side route registration (router/src/server.rs:L1633)
.route("/v1/chat/completions", post(chat_completions_v1))

Import

from openai import OpenAI

I/O Contract

Inputs

Name	Type	Required	Description
base_url	str	Yes	LoRAX server URL with /v1 path (e.g., "http://localhost:3000/v1")
api_key	str	Yes	API key (can be any string if no auth configured)

Outputs

Name	Type	Description
client	OpenAI	Configured OpenAI client pointing to LoRAX

Usage Examples

Basic Setup

from openai import OpenAI

# Point to local LoRAX server
client = OpenAI(
    base_url="http://localhost:3000/v1",
    api_key="lorax",  # Placeholder
)

# Use like standard OpenAI
response = client.chat.completions.create(
    model="my-org/my-lora-adapter",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is LoRA?"},
    ],
    max_tokens=200,
)
print(response.choices[0].message.content)

Related Pages

Implements Principle

• Principle:Predibase_Lorax_OpenAI_Compatible_Client