Implementation:Mlc ai Mlc llm DeepSeek Templates

Overview

The DeepSeek Templates module defines conversation templates for the DeepSeek family of large language models within the MLC LLM framework. Located at python/mlc_llm/conversation_template/deepseek.py, this file registers five distinct conversation templates with the global ConvTemplateRegistry: deepseek, deepseek_v2, deepseek_v3, deepseek_r1_qwen, and deepseek_r1_llama. Each template encapsulates the specific prompt formatting, role definitions, separator tokens, and stop conditions required by its corresponding model variant.

Purpose

Conversation templates are essential for structuring multi-turn dialogues between users and LLM assistants. Each DeepSeek model variant expects prompts formatted in a particular way. This module ensures that each variant is correctly handled by providing a pre-configured Conversation object registered under a unique name that the engine can look up at runtime.

File Location

python/mlc_llm/conversation_template/deepseek.py

Imports and Dependencies

from mlc_llm.protocol.conversation_protocol import Conversation, MessagePlaceholders
from .registry import ConvTemplateRegistry

The module depends on:

• Conversation -- the protocol dataclass that holds all template fields (name, system template, roles, separators, stop tokens, etc.).
• MessagePlaceholders -- an enum providing placeholder strings (e.g., MessagePlaceholders.SYSTEM.value) that are substituted at runtime with actual system messages.
• ConvTemplateRegistry -- the global registry where templates are stored and retrieved by name.

Registered Templates

deepseek (Original)

The original DeepSeek template uses simple role-based formatting with a colon separator between the role name and message content.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="deepseek",
        system_template=f"{MessagePlaceholders.SYSTEM.value}",
        system_message="",
        system_prefix_token_ids=[100000],
        roles={"user": "User", "assistant": "Assistant"},
        seps=["\n\n", "<｜end▁of▁sentence｜>"],
        role_content_sep=": ",
        role_empty_sep=":",
        stop_str=["<｜end▁of▁sentence｜>"],
        stop_token_ids=[100001],
    )
)

Field	Value	Description
name	deepseek	Template identifier
system_prefix_token_ids	[100000]	Prefix token prepended before the system message
roles	{"user": "User", "assistant": "Assistant"}	Simple text role labels
seps	["\n\n", "<｜end▁of▁sentence｜>"]	Separator after user turn (double newline) and assistant turn (EOS marker)
stop_token_ids	[100001]	Token ID that signals generation termination

deepseek_v2

The DeepSeek V2 template is structurally identical to the original DeepSeek template. It uses the same role labels, separators, and stop tokens. The separate registration under the name deepseek_v2 allows model configurations to reference it distinctly.

deepseek_v3

DeepSeek-V3 introduces a different prompt structure that uses special Unicode-based marker tokens for role boundaries rather than plain text labels.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="deepseek_v3",
        system_template=f"<｜begin▁of▁sentence｜>{MessagePlaceholders.SYSTEM.value}",
        system_message="You are Deepseek-V3, an AI assistant created exclusively by the Chinese "
        "Company DeepSeek. You'll provide helpful, harmless, and detailed responses to all "
        "user inquiries.",
        roles={"user": "<｜User｜>", "assistant": "<｜Assistant｜>"},
        seps=["", "<｜end▁of▁sentence｜>"],
        role_content_sep="",
        role_empty_sep="",
        stop_token_ids=[1],
    )
)

Key differences from V1/V2:

• The system_template is prefixed with <｜begin▁of▁sentence｜>.
• A non-empty system_message is provided by default, identifying the model as DeepSeek-V3.
• roles use special Unicode marker tokens (<｜User｜>, <｜Assistant｜>) instead of plain English words.
• role_content_sep and role_empty_sep are empty strings, meaning content follows the role marker directly.
• No system_prefix_token_ids field -- the BOS marker is embedded in the template string itself.
• stop_token_ids is [1].

deepseek_r1_qwen

This template supports the DeepSeek-R1-Distill-Qwen models, which are distilled versions of DeepSeek-R1 based on the Qwen architecture.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="deepseek_r1_qwen",
        system_template=f"<｜begin▁of▁sentence｜>{MessagePlaceholders.SYSTEM.value}",
        system_message="You are Deepseek-R1, an AI assistant created exclusively by the Chinese "
        "Company DeepSeek. You'll provide helpful, harmless, and detailed responses to all "
        "user inquiries.",
        roles={"user": "<｜User｜>", "assistant": "<｜Assistant｜>"},
        seps=["", "<｜end▁of▁sentence｜>"],
        role_content_sep="",
        role_empty_sep="",
        stop_token_ids=[151643],
    )
)

The template is structurally identical to deepseek_v3 except:

• The system_message identifies the model as "Deepseek-R1".
• stop_token_ids is [151643], which corresponds to the Qwen tokenizer's end-of-text token.

deepseek_r1_llama

This template supports the DeepSeek-R1-Distill-Llama models, which are distilled from DeepSeek-R1 based on the Llama architecture.

ConvTemplateRegistry.register_conv_template(
    Conversation(
        name="deepseek_r1_llama",
        system_template=f"<｜begin▁of▁sentence｜>{MessagePlaceholders.SYSTEM.value}",
        system_message="You are Deepseek-R1, an AI assistant created exclusively by the Chinese "
        "Company DeepSeek. You'll provide helpful, harmless, and detailed responses to all"
        " user inquiries.",
        roles={"user": "<｜User｜>", "assistant": "<｜Assistant｜>"},
        seps=["", "<｜end▁of▁sentence｜>"],
        role_content_sep="",
        role_empty_sep="",
        stop_token_ids=[128001],
    )
)

As the source code comment notes, this template is "exactly the same as DeepSeek-R1-Distill-Qwen, but different stop token." The stop_token_ids value is [128001], matching the Llama tokenizer's <|end_of_text|> token.

Template Comparison

Template Name	System Message	Role Format	Stop Token IDs	Prefix Token IDs
deepseek	(empty)	User: / Assistant:	[100001]	[100000]
deepseek_v2	(empty)	User: / Assistant:	[100001]	[100000]
deepseek_v3	DeepSeek-V3 identity	<｜User｜> / <｜Assistant｜>	[1]	(none)
deepseek_r1_qwen	DeepSeek-R1 identity	<｜User｜> / <｜Assistant｜>	[151643]	(none)
deepseek_r1_llama	DeepSeek-R1 identity	<｜User｜> / <｜Assistant｜>	[128001]	(none)

Relationship to Other Modules

• Template Registry -- All templates in this file are registered via ConvTemplateRegistry.register_conv_template().
• Conversation Protocol -- Each template is a Conversation instance defined in mlc_llm.protocol.conversation_protocol.
• The templates are loaded at module import time. The conversation template package's __init__.py imports this module, ensuring all templates are registered when the package is first used.