Implementation:Huggingface Optimum RunConfig

Knowledge Sources	Huggingface_Optimum
Domains	Configuration, Benchmarking
Last Updated	2026-02-15 00:00 GMT

Overview

Concrete tool for defining and validating benchmarking run configuration parameters as a structured dataclass hierarchy provided by the Huggingface Optimum library.

Description

This module defines a hierarchy of pydantic-compatible dataclasses that together form the RunConfig schema. The hierarchy includes:

• Calibration — Parameters for post-training calibration (method, num_samples, percentile settings)
• FrameworkArgs — Framework-specific settings (ONNX opset, optimization level)
• DatasetArgs — Dataset path, splits, data keys, and reference keys
• TaskArgs — Task-specific parameters (e.g., is_regression for text classification)
• BenchmarkTimeArgs — Time benchmark duration and warmup configuration
• Run — Core run parameters (model, task, quantization approach, dataset, framework)
• RunConfig — Full configuration combining all parameters with batch_sizes and input_lengths

Supporting enums include Frameworks, CalibrationMethods, and QuantizationApproach. The APIFeaturesManager class validates that tasks and model types are supported.

Usage

Use RunConfig to validate and structure parameters before launching a benchmarking run. The dataclass supports initialization from nested dictionaries for Python 3.8 compatibility.

Code Reference

Source Location

• Repository: Huggingface_Optimum
• File: optimum/utils/runs.py
• Lines: 1-300

Signature

@dataclass
class RunConfig(Run, _RunConfigDefaults, _RunConfigBase):
    """Parameters defining a run.

    A run evaluates a triplet (model, dataset, metric) with optimization
    parameters, comparing a transformers baseline and a model optimized
    with Optimum.
    """

    metrics: List[str]
    model_name_or_path: str
    task: str
    quantization_approach: QuantizationApproach
    dataset: DatasetArgs
    framework: Frameworks
    framework_args: FrameworkArgs
    operators_to_quantize: Optional[List[str]] = ["Add", "MatMul"]
    node_exclusion: Optional[List[str]] = ["layernorm", "gelu", "residual", "gather", "softmax"]
    per_channel: Optional[bool] = False
    calibration: Optional[Calibration] = None
    task_args: Optional[TaskArgs] = None
    aware_training: Optional[bool] = False
    max_eval_samples: Optional[int] = None
    time_benchmark_args: Optional[BenchmarkTimeArgs] = BenchmarkTimeArgs()
    batch_sizes: Optional[List[int]] = [4, 8]
    input_lengths: Optional[List[int]] = [128]

Import

from optimum.utils.runs import (
    RunConfig,
    Calibration,
    DatasetArgs,
    FrameworkArgs,
    TaskArgs,
    BenchmarkTimeArgs,
    APIFeaturesManager,
    QuantizationApproach,
    Frameworks,
    CalibrationMethods,
)

I/O Contract

Inputs (RunConfig)

Name	Type	Required	Description
model_name_or_path	str	Yes	Name of the model on the Hub
task	str	Yes	Task performed (e.g., text-classification, question-answering)
quantization_approach	QuantizationApproach	Yes	static or dynamic quantization
dataset	DatasetArgs	Yes	Dataset configuration with path, splits, keys
framework	Frameworks	Yes	Optimization framework (e.g., onnxruntime)
framework_args	FrameworkArgs	Yes	Framework-specific arguments
metrics	List[str]	Yes	Metrics to evaluate
batch_sizes	List[int]	No	Batch sizes for time benchmarking (default: [4, 8])
input_lengths	List[int]	No	Input lengths for time benchmarking (default: [128])
calibration	Calibration	No	Calibration config for static quantization

Outputs

Name	Type	Description
RunConfig instance	RunConfig	Validated configuration object with all parameters accessible as attributes

Usage Examples

Creating a RunConfig

from optimum.utils.runs import RunConfig

config = RunConfig(
    model_name_or_path="bert-base-uncased",
    task="text-classification",
    quantization_approach="dynamic",
    dataset={
        "path": "glue",
        "name": "sst2",
        "eval_split": "validation",
        "data_keys": {"primary": "sentence"},
        "ref_keys": ["label"],
    },
    framework="onnxruntime",
    framework_args={"opset": 13, "optimization_level": 1},
    metrics=["accuracy", "f1"],
    task_args={"is_regression": False},
    batch_sizes=[1, 4, 8],
    input_lengths=[64, 128],
)

Related Pages

• Environment:Huggingface_Optimum_Python_Core_Dependencies