Implementation:Mlc ai Mlc llm MLC Chat Config

Overview

The MLC Chat Config module defines the schema for the mlc-chat-config.json configuration file used by MLC LLM. Located at python/mlc_llm/protocol/mlc_chat_config.py (76 lines), it contains the MLCChatConfig Pydantic model that captures all model-level configuration including architecture, quantization, tokenizer, generation defaults, and conversation template information.

Purpose

This module provides the authoritative Python schema for the mlc-chat-config.json file that accompanies every compiled MLC LLM model. The config file is generated during model compilation and consumed at runtime by the MLC LLM engine. It bundles together model architecture details, quantization settings, tokenizer information, and generation defaults into a single structured configuration.

Key Components

System Default Constants

A dictionary defining fallback values for generation parameters when they are not explicitly set in the config:

MLC_CHAT_SYSTEM_DEFAULT = {
    "pad_token_id": 0,
    "bos_token_id": 1,
    "eos_token_id": 2,
    "temperature": 1.0,
    "presence_penalty": 0.0,
    "frequency_penalty": 0.0,
    "repetition_penalty": 1.0,
    "top_p": 1.0,
}

These defaults ensure sensible generation behavior when specific values are not provided.

MLCChatConfig Class

The main Pydantic BaseModel representing the full chat configuration schema:

class MLCChatConfig(BaseModel):
    """Fields in the dumped `mlc-chat-config.json` file."""

    version: str = MLC_CHAT_CONFIG_VERSION
    field_model_type: str = Field(alias="model_type")
    quantization: str
    field_model_config: Dict[str, Any] = Field(alias="model_config")
    vocab_size: int
    context_window_size: int
    sliding_window_size: int
    prefill_chunk_size: int
    attention_sink_size: int
    tensor_parallel_shards: int
    pipeline_parallel_stages: int = 1
    # ... generation and tokenizer fields ...
    conv_template: Conversation

Fields:

Field	Type	Default	Description
version	str	MLC_CHAT_CONFIG_VERSION	Config schema version for compatibility
model_type (alias)	str	(required)	Model architecture type (uses field_model_type internally to avoid Pydantic namespace conflict)
quantization	str	(required)	Quantization method identifier (e.g., "q4f16_1")
model_config (alias)	Dict[str, Any]	(required)	Model architecture configuration dictionary (uses field_model_config internally)
vocab_size	int	(required)	Vocabulary size
context_window_size	int	(required)	Maximum context window length
sliding_window_size	int	(required)	Sliding window attention size
prefill_chunk_size	int	(required)	Chunk size for prefill stage
attention_sink_size	int	(required)	Number of attention sink tokens
tensor_parallel_shards	int	(required)	Number of tensor parallelism shards
pipeline_parallel_stages	int	1	Number of pipeline parallelism stages
active_vocab_size	int	None	Active vocabulary size (subset of full vocab)
temperature	Optional[float]	None	Default sampling temperature
presence_penalty	Optional[float]	None	Default presence penalty
frequency_penalty	Optional[float]	None	Default frequency penalty
repetition_penalty	Optional[float]	None	Default repetition penalty
top_p	Optional[float]	None	Default top-p nucleus sampling value
tokenizer_files	List[str]	[]	List of tokenizer file paths
tokenizer_info	Dict[str, Any]	{}	Content of tokenizer.TokenizerInfo
conv_template	Conversation	(required)	The conversation template for prompt formatting
pad_token_id	Optional[int]	None	Padding token ID
bos_token_id	Optional[int]	None	Beginning-of-sequence token ID
eos_token_id	Optional[Union[int, List[int]]]	None	End-of-sequence token ID(s); supports single or multiple EOS tokens

Pydantic Field Aliases:

The fields model_type and model_config use Pydantic aliases because these names conflict with Pydantic's protected namespace. They are stored internally as field_model_type and field_model_config but serialized/deserialized with their alias names.

get_system_defaults_for_missing_fields Method

This method returns a dictionary of system default values for any generation-related fields that are None:

def get_system_defaults_for_missing_fields(self) -> Dict[str, Any]:
    res = {}
    for key, value in MLC_CHAT_SYSTEM_DEFAULT.items():
        if getattr(self, key) is None:
            res[key] = value
    return res

This implements a lazy default pattern: the config can be created with None values for optional generation fields, allowing them to be overridden by the user before finally applying system defaults. The method does not modify the config in place; it only returns the defaults that would need to be applied.

Dependencies

• pydantic -- For BaseModel and Field validation
• mlc_llm.support.constants.MLC_CHAT_CONFIG_VERSION -- The version constant for config schema versioning
• mlc_llm.protocol.conversation_protocol.Conversation -- The Conversation Protocol class for the conversation template

File Location

python/mlc_llm/protocol/mlc_chat_config.py