Implementation:Mlc ai Mlc llm Conversation Protocol

Overview

The Conversation Protocol module defines the standard conversation template system used throughout MLC LLM. It is located at python/mlc_llm/protocol/conversation_protocol.py (240 lines) and provides the data structures and logic for converting multi-turn conversations into model-consumable prompts. The module supports text messages, multimodal content (images), function calling placeholders, and configurable separator and role formatting.

Purpose

This module serves as the canonical representation of chat conversations within the MLC LLM serving infrastructure. It handles:

• Defining role-based message placeholders (system, user, assistant, tool, function)
• Managing conversation history as a list of role-content pairs
• Rendering the full conversation into a formatted prompt string (or list of mixed data types for multimodal inputs)
• Supporting diverse prompt formats used by different LLM families (e.g., [INST]...[/INST] style)

Key Components

MessagePlaceholders Enum

An enumeration that maps conversation roles to their corresponding placeholder strings in templates:

class MessagePlaceholders(Enum):
    SYSTEM = "{system_message}"
    USER = "{user_message}"
    ASSISTANT = "{assistant_message}"
    TOOL = "{tool_message}"
    FUNCTION = "{function_string}"

These placeholders are used in role templates and system templates and are replaced with actual content during prompt rendering.

Conversation Class

The core class, extending Pydantic's BaseModel, that defines the conversation template and holds conversation history.

Key Fields:

Field	Type	Description
name	Optional[str]	Optional name for the conversation template
system_template	str	Template string for the system prompt, containing the {system_message} placeholder
system_message	str	The actual system message content (without template formatting)
system_prefix_token_ids	Optional[List[int]]	Token IDs to prepend at the beginning of the tokenized prompt
add_role_after_system_message	bool	Whether to append user role and separator after the system message (default True)
roles	Dict[str, str]	Mapping of role identifiers to their display strings
role_templates	Dict[str, str]	Role-specific prompt templates with message placeholders
messages	List[Tuple[str, Optional[Union[str, List[Dict]]]]]	Conversation history as role-content pairs
seps	List[str]	Separator strings between messages (size 1 or 2)
role_content_sep	str	Separator between role label and content
role_empty_sep	str	Separator between role label and empty content
stop_str	List[str]	Stop strings for generation termination
stop_token_ids	List[int]	Stop token IDs for generation termination
function_string	str	The function schema string for function calling
use_function_calling	bool	Flag for whether function calling is active

Constructor:

The __init__ method initializes default role templates for user, assistant, and tool roles, which can be overridden by model-specific templates:

def __init__(self, role_templates: Optional[Dict[str, str]] = None, **kwargs):
    _role_templates: Dict[str, str] = {
        "user": MessagePlaceholders.USER.value,
        "assistant": MessagePlaceholders.ASSISTANT.value,
        "tool": MessagePlaceholders.TOOL.value,
    }
    if role_templates is not None:
        _role_templates.update(role_templates)
    super().__init__(role_templates=_role_templates, **kwargs)

Separator Validation:

A Pydantic field validator ensures that seps has exactly 1 or 2 entries:

@field_validator("seps")
@classmethod
def check_message_seps(cls, seps: List[str]) -> List[str]:
    if len(seps) == 0 or len(seps) > 2:
        raise ValueError("seps should have size 1 or 2.")
    return seps

When 1 separator is provided, it is used between all messages. When 2 are provided, seps[0] is used after user messages and seps[1] after assistant messages.

as_prompt Method

The as_prompt method is the primary rendering function that converts the conversation template and history into a list of prompt segments. It performs the following steps:

1. Substitutes the system message into the system template
2. Iterates over all messages, applying role prefixes, role templates, and separators
3. Handles multimodal content (text and image_url items) by producing ImageData objects via lazy import from mlc_llm.serve.data
4. Combines consecutive string segments via _combine_consecutive_messages
5. Substitutes the function string placeholder with the actual function definition

def as_prompt(self, config=None) -> List[Any]:
    from ..serve import data

    system_msg = self.system_template.replace(
        MessagePlaceholders.SYSTEM.value, self.system_message
    )
    message_list: List[Union[str, data.Data]] = []
    separators = list(self.seps)
    if len(separators) == 1:
        separators.append(separators[0])

    if system_msg != "":
        message_list.append(system_msg)

    for i, (role, content) in enumerate(self.messages):
        if role not in self.roles.keys():
            raise ValueError(f'Role "{role}" is not a supported role in {self.roles.keys()}')
        separator = separators[role == "assistant"]
        # ... message processing logic ...
    prompt = _combine_consecutive_messages(message_list)
    # ... function string substitution ...
    return prompt

The method uses role == "assistant" as a boolean index (0 or 1) to select the appropriate separator, which is a concise way of choosing between the two separator values.

Serialization Methods

def to_json_dict(self) -> Dict[str, Any]:
    return self.model_dump(by_alias=True, exclude_none=True)

@classmethod
def from_json_dict(cls: Type[T], json_dict: Dict[str, Any]) -> T:
    return Conversation.model_validate(json_dict)

These methods provide JSON serialization and deserialization using Pydantic's model dump and validation.

Helper Functions

_get_url_from_item: Extracts the URL from an image content item, supporting both string format and dict format (with a url key):

def _get_url_from_item(item: Dict) -> str:
    assert "image_url" in item
    if isinstance(item["image_url"], str):
        image_url = item["image_url"]
    elif isinstance(item["image_url"], dict):
        assert "url" in item["image_url"]
        image_url = item["image_url"]["url"]
    else:
        raise ValueError(...)
    return image_url

_combine_consecutive_messages: Merges adjacent string messages into a single string while preserving non-string (e.g., ImageData) items as separate list elements:

def _combine_consecutive_messages(messages: List[Any]) -> List[Any]:
    if len(messages) == 0:
        return []
    combined_messages = [messages[0]]
    for message in messages[1:]:
        if isinstance(message, str) and isinstance(combined_messages[-1], str):
            combined_messages[-1] += message
        else:
            combined_messages.append(message)
    return combined_messages

Prompt Format

The general prompt format produced by this module follows this pattern:

<<system>><<messages[0][0]>><<role_content_sep>><<messages[0][1]>><<seps[0]>>
           <<messages[1][0]>><<role_content_sep>><<messages[1][1]>><<seps[1]>>
           ...
           <<roles[1]>><<role_empty_sep>>

Where the last line represents the assistant role with empty content, signaling the model to begin generation.

Dependencies

• pydantic -- For data validation and serialization (BaseModel, Field, field_validator)
• mlc_llm.serve.data -- Lazy import for ImageData and Data types used in multimodal prompt construction

File Location

python/mlc_llm/protocol/conversation_protocol.py