Implementation:Mlc ai Mlc llm Template Registry

Overview

The Template Registry module defines the ConvTemplateRegistry class and registers several foundational conversation templates within MLC LLM. Located at python/mlc_llm/conversation_template/registry.py, this file serves as the central registration point for all conversation templates used throughout the framework. It also provides three built-in templates: chatml, chatml_nosystem, and LM.

Purpose

MLC LLM supports many different model families, each requiring a specific prompt format for multi-turn conversations. The ConvTemplateRegistry provides a global, static registry where templates are stored by name and can be retrieved by the engine at runtime. This design decouples template definitions (spread across model-specific modules like deepseek.py, llama.py, phi.py) from the engine code that consumes them.

File Location

python/mlc_llm/conversation_template/registry.py

Imports and Dependencies

from typing import Dict, Optional
from mlc_llm.protocol.conversation_protocol import Conversation, MessagePlaceholders

ConvTemplateRegistry Class

The ConvTemplateRegistry class uses a class-level dictionary as a global store. It has no instance state and exposes two static methods.

class ConvTemplateRegistry:
    """Global conversation template registry for preset templates."""

    _conv_templates: Dict[str, Conversation] = {}

    @staticmethod
    def register_conv_template(conv_template: Conversation, override: bool = False) -> None:
        """Register a new conversation template in the global registry.
        Using `override = True` to override the previously registered
        template with the same name.
        """
        name = conv_template.name
        if name is None:
            raise ValueError("The template to register should have non-None name.")
        if name in ConvTemplateRegistry._conv_templates and not override:
            raise ValueError(
                "The name of the template has been registered "
                f"for {ConvTemplateRegistry._conv_templates[name].model_dump_json(by_alias=True)}"
            )
        ConvTemplateRegistry._conv_templates[name] = conv_template

    @staticmethod
    def get_conv_template(name: str) -> Optional[Conversation]:
        """Return the conversation template specified by the given name,
        or None if the template is not registered.
        """
        return ConvTemplateRegistry._conv_templates.get(name, None)

register_conv_template

Parameter	Type	Description
conv_template	Conversation	The conversation template object to register. Must have a non-None name field.
override	bool	If True, allows overwriting a previously registered template with the same name. Defaults to False.

Validation behavior:

• Raises ValueError if the template's name is None.
• Raises ValueError if a template with the same name is already registered and override is False. The error message includes the JSON representation of the existing template.

get_conv_template

Parameter	Type	Description
name	str	The name of the template to retrieve.

Returns the Conversation object if found, or None if no template with the given name is registered.

Built-in Templates

The registry module registers three foundational templates that are not specific to any particular model family.

chatml

The ChatML format, widely used by many instruction-tuned models (including Qwen, OpenHermes, and others).

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="chatml",
        system_template=f"<|im_start|>system\n{MessagePlaceholders.SYSTEM.value}<|im_end|>\n",
        system_message=(
            "A conversation between a user and an LLM-based AI assistant. The "
            "assistant gives helpful and honest answers."
        ),
        roles={"user": "<|im_start|>user", "assistant": "<|im_start|>assistant"},
        seps=["<|im_end|>\n"],
        role_content_sep="\n",
        role_empty_sep="\n",
        stop_str=["<|im_end|>"],
        stop_token_ids=[2],
    )
)

Key characteristics:

• Uses <|im_start|> and <|im_end|> markers to delimit message boundaries.
• The system template wraps the system message between <|im_start|>system\n and <|im_end|>\n.
• A default system message is provided describing the assistant's behavior.

chatml_nosystem

A variant of ChatML that omits the system prompt entirely.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="chatml_nosystem",
        system_template=f"{MessagePlaceholders.SYSTEM.value}",
        system_message="",
        roles={"user": "<|im_start|>user", "assistant": "<|im_start|>assistant"},
        seps=["<|im_end|>\n"],
        role_content_sep="\n",
        role_empty_sep="\n",
        stop_str=["<|im_end|>"],
        stop_token_ids=[2],
    )
)

This template is identical to chatml except:

• The system_template is just the raw placeholder (no <|im_start|>system wrapper).
• The system_message is empty.

This is useful for models trained without system prompts or when the caller wants full control over the system message.

LM (Vanilla Language Model)

A minimal template for vanilla (non-instruction-tuned) language models used in pure text completion mode.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="LM",
        system_template=f"{MessagePlaceholders.SYSTEM.value}",
        system_message="",
        roles={"user": "", "assistant": ""},
        seps=[""],
        role_content_sep="",
        role_empty_sep="",
        stop_str=[],
        stop_token_ids=[2],
        system_prefix_token_ids=[1],
    )
)

Key characteristics:

• All role labels, separators, and content separators are empty strings -- no prompt formatting is applied.
• system_prefix_token_ids is [1] (BOS token).
• stop_str is empty; generation stops only on token ID 2 (EOS).
• This template passes input directly to the model without any role markers or structural formatting.

Template Comparison

Template	System Wrapper	System Message	Role Style	Stop Token IDs
chatml	im_start|>system....<|im_end|>	Helpful/honest assistant	im_start|>user / <|im_start|>assistant	[2]
chatml_nosystem	(none)	(empty)	im_start|>user / <|im_start|>assistant	[2]
LM	(none)	(empty)	(empty strings)	[2]

Design Pattern

The registry uses the Static Registry pattern:

• The _conv_templates dictionary is a class variable, not an instance variable, making it a process-wide singleton.
• Both methods are static methods, so no instantiation of ConvTemplateRegistry is needed.
• Templates are registered at module import time -- when registry.py is imported, the three built-in templates are immediately registered.
• Model-specific template modules (deepseek.py, llama.py, phi.py, etc.) import ConvTemplateRegistry from this module and call register_conv_template at their own import time.

Relationship to Other Modules

• DeepSeek Templates -- Imports and uses ConvTemplateRegistry to register DeepSeek model templates.
• Llama Templates -- Imports and uses ConvTemplateRegistry to register Llama model templates.
• Phi Templates -- Imports and uses ConvTemplateRegistry to register Phi model templates.
• Conversation Protocol (mlc_llm.protocol.conversation_protocol) -- Defines the Conversation dataclass and MessagePlaceholders enum used by all templates.
• Engine modules -- Call ConvTemplateRegistry.get_conv_template(name) to retrieve the appropriate template for a given model at runtime.