Implementation:Sgl project Sglang ModelConfig And LoadConfig

Knowledge Sources	SGLang
Domains	Quantization, Model_Optimization, Configuration
Last Updated	2026-02-10 00:00 GMT

Overview

Concrete tool for configuring model architecture and quantization settings using SGLang's ModelConfig and LoadConfig dataclasses.

Description

ModelConfig holds model architecture information including the model path, quantization method, HuggingFace config, and derived properties. LoadConfig specifies how to load and where to export the model, including ModelOpt-specific paths for checkpoints and exports. These are constructed by the Engine/Server during initialization, or manually for standalone quantization scripts.

Usage

Construct ModelConfig and LoadConfig when writing standalone quantization scripts. For standard Engine/Server usage, these are constructed automatically from ServerArgs.

Code Reference

Source Location

• Repository: sglang
• File: python/sglang/srt/configs/model_config.py (ModelConfig: L84-230)
• File: python/sglang/srt/configs/load_config.py (LoadConfig: L36-136)

Signature

class ModelConfig:
    def __init__(
        self,
        model_path: str,
        quantization: Optional[str] = None,
        # Additional fields from HuggingFace PretrainedConfig
        ...
    ):
        """Model architecture and quantization configuration."""

@dataclass
class LoadConfig:
    load_format: Union[str, LoadFormat, type] = LoadFormat.AUTO
    download_dir: Optional[str] = None
    model_loader_extra_config: Optional[dict] = None
    modelopt_export_path: Optional[str] = None
    modelopt_checkpoint_save_path: Optional[str] = None
    modelopt_checkpoint_restore_path: Optional[str] = None

Import

from sglang.srt.configs.model_config import ModelConfig
from sglang.srt.configs.load_config import LoadConfig

I/O Contract

Inputs

Name	Type	Required	Description
model_path	str	Yes	HuggingFace model ID or local path
quantization	Optional[str]	No	Quantization method (e.g., "modelopt_fp8", "modelopt_fp4")
modelopt_export_path	Optional[str]	No	Directory to export quantized model
modelopt_checkpoint_save_path	Optional[str]	No	Path to save quantization checkpoint

Outputs

Name	Type	Description
ModelConfig	ModelConfig	Model architecture and quantization configuration
LoadConfig	LoadConfig	Loading format and export path configuration

Usage Examples

ModelOpt Quantization Setup

from sglang.srt.configs.model_config import ModelConfig
from sglang.srt.configs.load_config import LoadConfig

model_config = ModelConfig(
    model_path="meta-llama/Llama-3.1-8B-Instruct",
    quantization="modelopt_fp8",
)

load_config = LoadConfig(
    modelopt_export_path="/tmp/quantized_model",
    modelopt_checkpoint_save_path="/tmp/quantized_checkpoint",
)

Related Pages

Implements Principle

• Principle:Sgl_project_Sglang_Model_Quantization_Configuration