Implementation:Predibase Lorax Conversation State Pattern

Knowledge Sources	LoRAX OpenAI Chat Guide
Domains	API_Design, Conversation_Management
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for implementing multi-turn conversations with a stateless LoRAX server, using client-side message accumulation as a pattern.

Description

This is a Pattern Doc rather than an API Doc. The server is stateless; conversation management is the client's responsibility. The server-side Message struct (router/src/lib.rs) defines the input format, and ChatMessage (in the response) provides the assistant's reply. The client accumulates messages and sends the full array with each request.

The Message struct supports text content, multi-modal content (image URLs), and tool call results.

Usage

Implement in your application code. No server-side configuration needed. Simply maintain a list of messages and append each exchange.

Code Reference

Source Location

• Repository: LoRAX
• File: router/src/lib.rs
• Lines: 590-608 (Message input), 1049-1057 (ChatMessage response)

Signature

// Input message format
pub(crate) struct Message {
    pub role: String,
    pub content: MessageContent,  // Text or multimodal chunks
    pub name: Option<String>,
}

pub(crate) enum MessageContent {
    SingleText(String),
    MultipleChunks(Vec<MessageChunk>),
}

// Response message format
pub struct ChatMessage {
    pub role: Option<String>,       // "assistant"
    pub content: Option<String>,    // Generated text
    pub tool_calls: Option<Vec<ToolCall>>,
}

Import

# Client-side pattern using OpenAI SDK
from openai import OpenAI

I/O Contract

Inputs

Name	Type	Required	Description
messages	List[Message]	Yes	Full conversation history (client-maintained)

Outputs

Name	Type	Description
ChatMessage	object	Assistant response to append to client history

Usage Examples

Multi-Turn Conversation

from openai import OpenAI

client = OpenAI(base_url="http://localhost:3000/v1", api_key="x")
messages = [{"role": "system", "content": "You are a Python tutor."}]

# Turn 1
messages.append({"role": "user", "content": "What is a list comprehension?"})
response = client.chat.completions.create(model="my-adapter", messages=messages)
assistant_reply = response.choices[0].message.content
messages.append({"role": "assistant", "content": assistant_reply})

# Turn 2 - includes full history
messages.append({"role": "user", "content": "Can you give an example?"})
response = client.chat.completions.create(model="my-adapter", messages=messages)
assistant_reply = response.choices[0].message.content
messages.append({"role": "assistant", "content": assistant_reply})

# messages now has 5 entries: system + 2 user + 2 assistant

Related Pages

Implements Principle

• Principle:Predibase_Lorax_Stateless_Conversation_Management