Implementation:Mlc ai Mlc llm CLI Help

Overview

The CLI Help module provides all help message strings for MLC LLM's command-line interface arguments. Located at python/mlc_llm/interface/help.py, this file contains a single dictionary named HELP that maps argument names to their human-readable descriptions. These descriptions are used by the argument parser to display usage information when users invoke MLC LLM CLI commands with --help.

Purpose

By centralizing all CLI help text in one file, the project maintains consistent and easily updatable documentation for every command-line argument across all MLC LLM subcommands (compile, quantize, serve, calibrate, package, etc.). Rather than scattering help strings throughout the codebase, each CLI module references HELP["argument_name"].

File Location

python/mlc_llm/interface/help.py

Structure

The entire module consists of a single dictionary:

HELP = {
    "config": """...""".strip(),
    "quantization": """...""".strip(),
    "model": """...""".strip(),
    # ... additional entries
}

Each value is a multi-line string (using triple quotes) with .strip() applied to remove leading/trailing whitespace.

Help Entries

The following table summarizes all entries in the HELP dictionary, organized by functional area.

Model Configuration

Key	Description
config	Path to a HuggingFace model directory, a config.json file, or a pre-defined model architecture name. Supports HuggingFace format model configurations.
model	Path to mlc-chat-config.json, an MLC model directory, or a HuggingFace repository link pointing to a compiled MLC model.
model_type	Model architecture (e.g., "llama"). Inferred from mlc-chat-config.json if not explicitly set.
model_lib	Full path to the model library file (e.g., .so). If unspecified, the system searches or compiles via JIT.
conv_template	Conversation template name. Depends on how the model was fine-tuned. Use "LM" for vanilla base models.

Compilation

Key	Description
quantization	The quantization mode for compilation. Inferred from the model if not provided.
device_compile	GPU device to compile to. Inferred from locally available GPUs if not set.
host	The LLVM target triple (e.g., arm64-apple-ios, aarch64-linux-android, wasm32-unknown-unknown-wasm).
opt	Optimization flags. Supports presets O0-O3 or explicit knobs (e.g., --opt="cublas_gemm=1;cudagraph=0").
output_compile	Output file path. Suffix determines format: .so/.dylib/.dll (shared lib), .tar (objects), .wasm (web assembly).
system_lib_prefix	Prefix added to all exported symbols. Useful when compiling multiple models into one library to avoid symbol conflicts.
debug_dump	Directory for storing compiler IR dumps during compilation phases. Disabled by default.

Model Config Overrides

Key	Description
context_window_size	Maximum sequence length. Defaults to context_window_size or max_position_embeddings from config.json.
sliding_window_size	Sliding window size for SWA models (experimental). Currently most useful for Mistral.
prefill_chunk_size	Chunk size during prefilling (experimental). Defaults to sliding window size or max sequence length.
attention_sink_size	Number of stored attention sinks (experimental). Defaults to 4. Only supported on Mistral.
max_batch_size	Maximum batch size for KV cache concurrent support.
tensor_parallel_shards	Number of shards for tensor parallelism multi-GPU inference.
pipeline_parallel_stages	Number of pipeline stages for pipeline parallelism.
disaggregation	Whether to enable disaggregation during model compilation.
overrides	Semicolon-delimited model config override string for compile-time settings.
modelconfig_overrides	Similar to overrides but with a slightly different set of supported fields.

Quantization

Key	Description
source	Path to original model weights. Inferred from config if missing.
source_format	Format of source model weights. Inferred from config if missing.
device_quantize	Device used for quantization (e.g., "cuda" or "cuda:0"). Auto-detected if not specified.
output_quantize	Output directory for quantized weights. Produces params_shard_*.bin and tensor-cache.json.

Serving

Key	Description
device_deploy	Device for model deployment (e.g., "cuda:0"). Auto-detected if not specified.
mode_serve	Engine mode: "local" (low concurrency, batch size 4), "interactive" (single request), or "server" (maximize GPU utilization). Default is "local".
max_total_sequence_length_serve	Total KV cache token capacity. Auto-estimated from vRAM if not specified.
prefill_chunk_size_serve	Maximum tokens per prefill pass. Defaults to model config value.
max_history_size_serve	Maximum history length for RNN state rollback. Defaults to 1. Not used by KV cache models.
gpu_memory_utilization_serve	Fraction of GPU memory used (0-1). Defaults to 0.85.
enable_tracing_serve	Enable Chrome Tracing. Traces can be dumped via POST to /debug/dump_event_trace.
additional_models_serve	Additional model paths for speculative decoding. Supports optional model lib paths.
overrides_serve	Semicolon-delimited engine config override string for serve-time settings.
prefill_mode	Prefill mode: "chunked" (basic chunked prefill) or "hybrid" (hybrid prefill / split-fuse).

Speculative Decoding

Key	Description
speculative_mode_serve	Mode: "disable", "small_draft", "eagle", or "medusa". Default is "disable".
spec_draft_length_serve	Number of draft tokens per speculative proposal. 0 enables adaptive mode. Default is 0.

Prefix Cache

Key	Description
prefix_cache_mode_serve	Mode: "disable" or "radix" (paged radix tree). Default is "radix".
prefix_cache_max_num_recycling_seqs_serve	Maximum sequences in prefix cache. 0 disables, -1 for infinite capacity. Defaults to max batch size.

Calibration

Key	Description
calibration_dataset	Path to the calibration dataset.
num_calibration_samples	Number of samples for calibration.
output_calibration	Output directory for calibration parameters.
seed_calibrate	Random seed for sampling the calibration dataset.

Text Generation

Key	Description
prompt	The prompt for text generation.
generate_length	Target length of generated text.

Packaging

Key	Description
config_package	Path to mlc-package-config.json for package builds.
mlc_llm_source_dir	Source code path to MLC LLM.
output_package	Output directory for package build outputs.

Disaggregated Serving

Key	Description
pd_balance_factor	Fraction of prefill to move to the decode engine (e.g., 0.1 means last 10% of tokens prefilled by decode engine).
output_gen_mlc_chat_config	Output directory for generated configurations including mlc-chat-config.json and tokenizer config.

Usage Pattern

Other modules in the codebase reference this dictionary when setting up argument parsers:

from mlc_llm.interface.help import HELP

parser.add_argument("--model", type=str, help=HELP["model"])
parser.add_argument("--device", type=str, help=HELP["device_deploy"])

Relationship to Other Modules

• Compiler Flags (mlc_llm.interface.compiler_flags) -- The opt and overrides help entries describe flags parsed by OptimizationFlags and ModelConfigOverride.
• Calibrate Interface (mlc_llm.interface.calibrate) -- The calibration-related help entries (calibration_dataset, num_calibration_samples, etc.) correspond to parameters of the calibrate() function.
• CLI Entrypoints -- Various CLI subcommand modules (compile, quantize, serve, calibrate, package, gen-config, etc.) import from this dictionary to populate their --help output.