Implementation:Predibase Lorax Prepare Chat Input

Knowledge Sources	LoRAX
Domains	NLP, Prompt_Engineering
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for converting OpenAI chat messages into model-specific prompts provided by the prepare_chat_input function and ChatTemplateRenderer.

Description

The prepare_chat_input() function in router/src/server.rs orchestrates chat input preparation. It handles tool/function calling integration, response format extraction, and delegates to Infer::apply_chat_template() which uses ChatTemplateRenderer::apply() backed by the minijinja Jinja2 engine. The renderer loads the chat template from the model's tokenizer configuration.

Usage

Called internally by the chat_completions_v1 handler. Not called directly by users.

Code Reference

Source Location

• Repository: LoRAX
• File: router/src/server.rs (Lines: 421-463)
• File: router/src/infer.rs (Lines: 39-147)

Signature

// router/src/server.rs
fn prepare_chat_input(
    infer: &Infer,
    response_format: Option<ResponseFormat>,
    tools: Option<Vec<Tool>>,
    tool_choice: ToolChoice,
    tool_prompt: Option<String>,
    guideline: Option<String>,
    messages: Vec<Message>,
) -> Result<(String, Option<ResponseFormat>, bool), InferError>;

// router/src/infer.rs
impl Infer {
    pub(crate) fn apply_chat_template(
        &self,
        guideline: Option<String>,
        messages: Vec<Message>,
        tools_and_prompt: Option<(Vec<Tool>, String)>,
    ) -> Result<String, InferError>;
}

// ChatTemplateRenderer
pub(crate) struct ChatTemplateRenderer {
    template: minijinja::Environment<'static>,
    bos_token: Option<String>,
    eos_token: Option<String>,
}

impl ChatTemplateRenderer {
    pub(crate) fn apply(
        &self,
        guideline: Option<&str>,
        messages: &[Message],
        tools_and_prompt: Option<(&[Tool], &str)>,
    ) -> Result<String, InferError>;
}

Import

// Internal modules
use crate::infer::Infer;

I/O Contract

Inputs

Name	Type	Required	Description
messages	Vec[Message]	Yes	Chat messages with role and content
tools	Option[Vec[Tool]]	No	Function calling tool definitions
tool_choice	ToolChoice	No	Tool selection strategy
guideline	Option[String]	No	System guideline for template
response_format	Option[ResponseFormat]	No	JSON schema constraint

Outputs

Name	Type	Description
prompt	String	Rendered prompt string from Jinja template
response_format	Option[ResponseFormat]	Possibly modified response format
using_tools	bool	Whether tool calling is active

Usage Examples

How It Works Internally

# What happens when you call:
client.chat.completions.create(
    model="my-adapter",
    messages=[
        {"role": "system", "content": "You are a code assistant."},
        {"role": "user", "content": "Write hello world in Python"},
    ],
)

# 1. ChatCompletionRequest is deserialized
# 2. prepare_chat_input() is called
# 3. Messages are rendered through the model's Jinja template:
#    "<s>[INST] You are a code assistant.\n\nWrite hello world in Python [/INST]"
# 4. This prompt string is sent to the inference engine

Related Pages

Implements Principle

• Principle:Predibase_Lorax_Chat_Template_Rendering

Requires Environment

• Environment:Predibase_Lorax_Python_Server_Dependencies