Implementation:Mlc ai Mlc llm Auto Config

Overview

python/mlc_llm/support/auto_config.py provides helper functions for automatically detecting and resolving model configuration files, model types, and quantization schemes. These functions form the configuration detection pipeline used during model compilation and serving, resolving paths from various input formats (preset names, HF URLs, local paths) and inferring model metadata when set to "auto".

Location

• File: python/mlc_llm/support/auto_config.py
• Module: mlc_llm.support.auto_config
• Lines: 193

Module-Level Constants

logger = logging.getLogger(__name__)
FOUND = green("Found")

Uses the MLC LLM custom logging module and a styled "Found" label (displayed in green) for consistent log output.

Functions

detect_mlc_chat_config

def detect_mlc_chat_config(mlc_chat_config: str) -> Path:

Detects and returns the path to mlc-chat-config.json.

Input resolution flow:

1. HF URL or HTTP URL: If the input starts with "HF://" or "http", the weights are downloaded and cached via download_and_cache_mlc_weights.
2. Preset model name: If the input matches a key in MODEL_PRESETS, the preset configuration is written to a temporary JSON file with an added model_preset_tag field.
3. Local path: Otherwise, treated as a local filesystem path.

Path validation:

• Raises ValueError if the resolved path does not exist.
• If the path is a directory, looks for mlc-chat-config.json inside it.
• If the path is a file, uses it directly.

if mlc_chat_config_path.is_dir():
    mlc_chat_config_json_path = mlc_chat_config_path / "mlc-chat-config.json"
    if not mlc_chat_config_json_path.exists():
        raise ValueError(f"Fail to find mlc-chat-config.json under {mlc_chat_config_path}.")
else:
    mlc_chat_config_json_path = mlc_chat_config_path

detect_config

def detect_config(config: str) -> Path:

Detects and returns the path to config.json (the standard model configuration file).

Input resolution flow:

1. Preset model name: If the input matches a key in MODEL_PRESETS, writes the preset to a temporary JSON file.
2. Local path: Otherwise, treats the input as a filesystem path.

Path validation:

• Raises ValueError if the path does not exist.
• If the path is a directory, looks for config.json inside it.
• If the path is a file, uses it directly.

Unlike detect_mlc_chat_config, this function does not handle HF or HTTP URLs -- it only supports preset names and local paths.

detect_model_type

def detect_model_type(model_type: str, config: Path) -> "Model":

Detects the model architecture type from the configuration file.

Logic:

1. If model_type is "auto", reads config.json and extracts the "model_type" field (checking both top-level and nested "model_config").
2. Applies a special mapping: "mixformer-sequential" is mapped to "phi-msft".
3. Looks up the model type in the MODELS registry.
4. Raises ValueError if the model type is not found in the configuration or registry.

if model_type == "auto":
    with open(config, "r", encoding="utf-8") as config_file:
        cfg = json.load(config_file)
    if "model_type" not in cfg and (
        "model_config" not in cfg or "model_type" not in cfg["model_config"]
    ):
        raise ValueError(
            f"'model_type' not found in: {config}. "
            f"Please explicitly specify `--model-type` instead."
        )
    model_type = cfg["model_type"] if "model_type" in cfg else cfg["model_config"]["model_type"]
if model_type in ["mixformer-sequential"]:
    model_type = "phi-msft"

Returns: A Model object from the MODELS registry.

detect_quantization

def detect_quantization(quantization_arg: str, config: Path) -> "Quantization":

Detects the quantization scheme from the configuration file or an explicit argument.

Logic:

1. If quantization_arg is not None, uses it to look up the scheme in the QUANTIZATION registry.
2. Otherwise, reads the "quantization" field from the config file.
3. Raises ValueError if neither source provides a quantization value.

with open(config, "r", encoding="utf-8") as config_file:
    cfg = json.load(config_file)
if quantization_arg is not None:
    quantization = QUANTIZATION[quantization_arg]
elif "quantization" in cfg:
    quantization = QUANTIZATION[cfg["quantization"]]
else:
    raise ValueError(
        f"'quantization' not found in: {config}. "
        f"Please explicitly specify `--quantization` instead."
    )

Returns: A Quantization object from the QUANTIZATION registry.

Function Summary

Function	Input	Output	Config File
detect_mlc_chat_config	Model path/name/URL	Path	mlc-chat-config.json
detect_config	Model path/name	Path	config.json
detect_model_type	Model type string + config path	Model	config.json
detect_quantization	Quantization arg + config path	Quantization	config.json or mlc-chat-config.json

Dependencies

• json: For reading and writing JSON config files.
• tempfile: For creating temporary config files from presets.
• pathlib.Path: For filesystem path operations.
• mlc_llm.model.MODEL_PRESETS: Registry of preset model configurations.
• mlc_llm.model.MODELS: Registry mapping model type strings to Model objects.
• mlc_llm.quantization.QUANTIZATION: Registry mapping quantization scheme names to Quantization objects.
• mlc_llm.support.download_cache: For downloading and caching weights from HF URLs.
• mlc_llm.support.logging: Custom logging module.
• mlc_llm.support.style: For styled terminal output (bold, green).

Design Notes

• Preset model names are resolved by writing their configuration to temporary files, allowing the rest of the pipeline to work with file paths uniformly.
• The model_preset_tag field is injected into preset configs to track which preset was used.
• The "mixformer-sequential" to "phi-msft" mapping handles legacy Phi model naming from Microsoft.
• Both detect_model_type and detect_quantization prefer explicit arguments over config file values, following a convention where CLI flags override file-based configuration.