Implementation:Mlc ai Mlc llm Phi Templates

Overview

The Phi Templates module defines conversation templates for the Microsoft Phi family of models within MLC LLM. Located at python/mlc_llm/conversation_template/phi.py, this file registers four conversation templates: phi-2, phi-3, phi-3-vision, and phi-4. Each template captures the specific prompt formatting, role markers, system prompts, and stop conditions appropriate for its corresponding Phi model variant.

Purpose

Microsoft's Phi models use different prompt formats across generations. Phi-2 uses simple role labels ("Instruct" / "Output"), while Phi-3 and Phi-4 adopt XML-like tag tokens (<|system|>, <|user|>, <|assistant|>). This module ensures correct prompt construction for each variant during inference.

File Location

python/mlc_llm/conversation_template/phi.py

Imports and Dependencies

from mlc_llm.protocol.conversation_protocol import Conversation, MessagePlaceholders
from .registry import ConvTemplateRegistry

Registered Templates

phi-2

Phi-2 uses a simple "Instruct" / "Output" role labeling scheme with colon separators.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="phi-2",
        system_template=f"{MessagePlaceholders.SYSTEM.value}",
        system_message="",
        roles={"user": "Instruct", "assistant": "Output"},
        seps=["\n"],
        role_content_sep=": ",
        role_empty_sep=":",
        stop_str=["<|endoftext|>"],
        stop_token_ids=[50256],
    )
)

Key characteristics:

• roles map user to Instruct and assistant to Output, reflecting Phi-2's training format.
• seps is a single newline, producing compact multi-turn formatting.
• stop_token_ids is [50256], the GPT-2 tokenizer's <|endoftext|> token.
• No system prefix token IDs or system message are set.

phi-3

Phi-3 introduces structured tag-based tokens for role demarcation with a default system safety prompt.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="phi-3",
        system_template=f"<|system|>\n{MessagePlaceholders.SYSTEM.value}",
        system_message="You are a helpful digital assistant. Please provide safe, "
        "ethical and accurate information to the user.",
        roles={"user": "<|user|>", "assistant": "<|assistant|>"},
        seps=["<|end|>\n"],
        role_content_sep="\n",
        role_empty_sep="\n",
        system_prefix_token_ids=[1],
        stop_str=["<|endoftext|>"],
        stop_token_ids=[2, 32000, 32001, 32007],
    )
)

Key characteristics:

• system_template wraps the system message with the <|system|> tag.
• A system_message about safety and ethics is included by default.
• roles use <|user|> and <|assistant|> tag tokens.
• seps is <|end|>\n -- each turn ends with the end tag plus a newline.
• stop_token_ids includes four IDs: [2, 32000, 32001, 32007].
• system_prefix_token_ids is [1] (BOS token).

phi-3-vision

The Phi-3-Vision template is used for the multimodal variant of Phi-3. It shares the same role and separator structure as Phi-3 but has no system message or system template wrapper.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="phi-3-vision",
        system_template=f"{MessagePlaceholders.SYSTEM.value}",
        system_message="",
        roles={"user": "<|user|>", "assistant": "<|assistant|>"},
        seps=["<|end|>\n"],
        role_content_sep="\n",
        role_empty_sep="\n",
        system_prefix_token_ids=[1],
        stop_str=["<|endoftext|>"],
        stop_token_ids=[2, 32000, 32001, 32007],
    )
)

The only difference from phi-3 is:

• system_template is just the placeholder (no <|system|> wrapper).
• system_message is empty.

phi-4

Phi-4 uses the same tag-based scheme as Phi-3 but with a different tokenizer vocabulary, resulting in different token IDs for special tokens.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="phi-4",
        system_template=f"<|system|>\n{MessagePlaceholders.SYSTEM.value}",
        system_message="You are a helpful digital assistant. Please provide safe, "
        "ethical and accurate information to the user.",
        roles={"user": "<|user|>", "assistant": "<|assistant|>"},
        seps=["<|end|>\n"],
        role_content_sep="\n",
        role_empty_sep="\n",
        system_prefix_token_ids=[200022],
        stop_str=["<|endoftext|>", "<|end|>"],
        stop_token_ids=[199999, 200020],
    )
)

Key differences from Phi-3:

• system_prefix_token_ids is [200022] (the <|system|> token ID in Phi-4's tokenizer).
• stop_str includes both <|endoftext|> and <|end|>.
• stop_token_ids are [199999, 200020], corresponding to <|endoftext|> and <|end|> in Phi-4's tokenizer.

Template Comparison

Template	System Message	Role Style	Stop Token IDs	Prefix Token IDs
phi-2	(empty)	Instruct: / Output:	[50256]	(none)
phi-3	Safety/ethics prompt	user|> / <|assistant|>	[2, 32000, 32001, 32007]	[1]
phi-3-vision	(empty)	user|> / <|assistant|>	[2, 32000, 32001, 32007]	[1]
phi-4	Safety/ethics prompt	user|> / <|assistant|>	[199999, 200020]	[200022]

Relationship to Other Modules

• Template Registry -- All templates are registered via ConvTemplateRegistry.register_conv_template().
• Conversation Protocol -- Templates instantiate the Conversation dataclass from mlc_llm.protocol.conversation_protocol.
• This module is imported at package initialization time, ensuring all Phi templates are available when the conversation template subsystem is loaded.