Implementation:InternLM Lmdeploy Calibrate

Knowledge Sources	LMDeploy
Domains	Quantization, Data_Processing
Last Updated	2026-02-07 15:00 GMT

Overview

Concrete tool for preparing calibration datasets and collecting activation statistics needed for model quantization provided by the LMDeploy library.

Description

The calibrate() function loads a text dataset, tokenizes it with the model's tokenizer, and collects activation statistics by running forward passes. It is called internally by both auto_awq() and smooth_quant() but can also be used independently via the llm-compressor workflow.

Usage

Usually called internally by auto_awq() or smooth_quant(). Use directly when implementing custom quantization workflows or when using the llm-compressor integration.

Code Reference

Source Location

• Repository: lmdeploy
• File: lmdeploy/lite/apis/calibrate.py
• Lines: L230-327

Signature

def calibrate(model: str,
              calib_dataset: str = 'wikitext2',
              calib_samples: int = 128,
              calib_seqlen: int = 2048,
              work_dir: str = './work_dir',
              search_scale: bool = False,
              batch_size: int = 1,
              w_bits: int = 4,
              w_sym: bool = False,
              w_group_size: int = 128,
              device: str = 'cuda',
              dtype: str = 'auto',
              revision: str = None,
              download_dir: str = None) -> Tuple:

Import

from lmdeploy.lite.apis.calibrate import calibrate

I/O Contract

Inputs

Name	Type	Required	Description
model	str	Yes	Model path for tokenizer and forward passes
calib_dataset	str	No	Dataset name (default: 'wikitext2')
calib_samples	int	No	Number of calibration samples (default: 128)
calib_seqlen	int	No	Sequence length for calibration (default: 2048)

Outputs

Name	Type	Description
vl_model	object	Vision-language model component (if applicable)
model	object	Loaded model with calibration hooks
tokenizer	object	Model tokenizer
work_dir	Path	Output directory with saved statistics

Usage Examples

from lmdeploy.lite.apis.calibrate import calibrate

# Calibrate for AWQ quantization
vl_model, model, tokenizer, work_dir = calibrate(
    model='internlm/internlm2_5-7b-chat',
    calib_dataset='wikitext2',
    calib_samples=128,
    calib_seqlen=2048,
    work_dir='./calibration_output'
)

Related Pages

Implements Principle

• Principle:InternLM_Lmdeploy_Calibration_Dataset_Preparation

Requires Environment

• Environment:InternLM_Lmdeploy_Python_Dependencies