Principle:InternLM Lmdeploy Calibration Dataset Preparation

Knowledge Sources	AWQ SmoothQuant LMDeploy
Domains	Quantization, Data_Processing
Last Updated	2026-02-07 15:00 GMT

Overview

A data preparation process that collects representative text samples and computes activation statistics needed for quantization-aware weight compression.

Description

Calibration Dataset Preparation is the prerequisite step for both AWQ and SmoothQuant quantization. The process:

1. Loads a representative text dataset (default: WikiText-2)
2. Tokenizes samples to a fixed sequence length using the model's tokenizer
3. Runs forward passes through the model to collect activation statistics
4. Saves statistics (activation ranges, outlier magnitudes) for the quantization step

The quality of calibration data directly impacts quantization accuracy. The dataset should be representative of the model's intended use case.

Usage

Required before running auto_awq or smooth_quant. Usually handled internally by the quantization CLI commands. Override the default dataset when quantizing domain-specific models.

Theoretical Basis

Calibration collects activation statistics needed for quantization parameter estimation:

# Abstract calibration process
for batch in calibration_data:
    activations = model.forward(batch, collect_stats=True)
    for layer in model.layers:
        stats[layer].update(
            max_activation=max(activations[layer]),
            mean_activation=mean(activations[layer])
        )

The statistics determine:

• AWQ: Which weight channels are salient (high activation magnitude)
• SmoothQuant: The smoothing factor per channel (activation/weight ratio)

Related Pages

Implemented By

• Implementation:InternLM_Lmdeploy_Calibrate