Principle:InternLM Lmdeploy Chat Template Formatting
| Knowledge Sources | |
|---|---|
| Domains | NLP, Prompt_Engineering |
| Last Updated | 2026-02-07 15:00 GMT |
Overview
A template system that converts raw user messages into model-specific prompt formats with system prefixes, role markers, and special tokens.
Description
Chat Template Formatting addresses the critical requirement that each LLM family expects prompts in a specific format. A message like "Hello, how are you?" must be wrapped with model-specific tokens:
- System prefix: Sets the model persona or instructions
- User marker: Indicates the start of user input (e.g., <|User|>, [INST])
- Assistant marker: Indicates where generation should begin
- End-of-turn tokens: Signals conversation turn boundaries
- Image tokens: Placeholder positions for vision-language models
The template system uses a registry pattern where templates are registered by model name and looked up automatically based on the loaded model. It supports both structured (OpenAI-format message lists) and unstructured (plain string) inputs.
Usage
Use this principle whenever converting user prompts into model-consumable format. This happens automatically inside the Pipeline but can be invoked directly for custom prompt engineering or debugging tokenization issues.
Theoretical Basis
Chat templates implement a format string pattern with role-based substitution:
# Abstract template application
def format_prompt(template, messages):
result = template.system_prefix + template.system_message
for msg in messages:
if msg.role == "user":
result += template.user_marker + msg.content + template.end_of_human
elif msg.role == "assistant":
result += template.assistant_marker + msg.content + template.end_of_assistant
result += template.assistant_marker # Prompt model to generate
return result