Implementation:Huggingface Optimum ExporterConfig Validation

Field	Value
Page Type	Implementation
Source Repository	https://github.com/huggingface/optimum
Source Files	optimum/exporters/base.py, optimum/exporters/error_utils.py
Domains	NLP, Computer_Vision, Export
Last Updated	2026-02-15 00:00 GMT

Overview

This implementation defines the validation tolerance attributes on ExporterConfig and the specialized error classes used to report validation failures. Together they form the validation framework that ensures exported models produce numerically equivalent outputs to the original.

API Reference

Tolerance Attributes on ExporterConfig

Source location: optimum/exporters/base.py, lines 77-112

The ExporterConfig abstract base class defines the following class-level attributes controlling validation behavior:

Attribute	Type	Default	Description
ATOL_FOR_VALIDATION	Union[float, Dict[str, float]]	1e-5	The absolute tolerance for output comparison. Can be a single float applied to all outputs, or a dictionary mapping task names to tolerance values.
ATOL_FOR_VALIDATION_FOR_EACH_OUTPUT	Dict[str, float]	(empty)	Per-output tolerance overrides. Maps specific output names to custom tolerance values.
NORMALIZED_CONFIG_CLASS	Type	None	The normalized config class used to extract model dimensions for dummy input generation.
DUMMY_INPUT_GENERATOR_CLASSES	Tuple[Type]	()	Tuple of DummyInputGenerator subclasses used to produce validation inputs.

Source excerpt:

class ExporterConfig(ABC):
    """
    Base class describing metadata on how to export the model.

    Class attributes:
    - NORMALIZED_CONFIG_CLASS -- specifying how to normalize the model config.
    - DUMMY_INPUT_GENERATOR_CLASSES -- specifying how to create dummy inputs.
    - ATOL_FOR_VALIDATION -- absolute tolerance value for model conversion validation.
    - MIN_TORCH_VERSION -- minimum torch version supporting the export.
    - MIN_TRANSFORMERS_VERSION -- minimum transformers version supporting the export.
    """

    NORMALIZED_CONFIG_CLASS = None
    DUMMY_INPUT_GENERATOR_CLASSES = ()
    ATOL_FOR_VALIDATION: Union[float, Dict[str, float]] = 1e-5
    MIN_TORCH_VERSION = GLOBAL_MIN_TORCH_VERSION
    MIN_TRANSFORMERS_VERSION = GLOBAL_MIN_TRANSFORMERS_VERSION

Subclasses override these attributes to customize validation behavior for specific model architectures. For example, a vision model config might set:

class ViTOnnxConfig(OnnxConfig):
    ATOL_FOR_VALIDATION = 1e-4  # Relaxed tolerance for vision models

Common Output Mapping

Source location: optimum/exporters/base.py, lines 113-140

The _TASK_TO_COMMON_OUTPUTS class attribute maps task names to expected output names, used during validation to verify that the exported model produces the correct outputs:

_TASK_TO_COMMON_OUTPUTS = {
    "audio-classification": ["logits"],
    "automatic-speech-recognition": ["logits"],
    "depth-estimation": ["predicted_depth"],
    "feature-extraction": ["last_hidden_state"],
    "fill-mask": ["logits"],
    "image-classification": ["logits"],
    "image-segmentation": ["logits"],
    "multiple-choice": ["logits"],
    "object-detection": ["logits", "pred_boxes"],
    "question-answering": ["start_logits", "end_logits"],
    "semantic-segmentation": ["logits"],
    "text2text-generation": ["logits"],
    "text-classification": ["logits"],
    "text-generation": ["logits"],
    "token-classification": ["logits"],
    "zero-shot-image-classification": ["logits_per_image", "logits_per_text",
                                        "text_embeds", "image_embeds"],
    "zero-shot-object-detection": ["logits", "pred_boxes",
                                    "text_embeds", "image_embeds"],
}

Error Classes

Source location: optimum/exporters/error_utils.py, lines 15-39

The validation framework uses specialized error classes to report different types of validation failures:

Error Class	Base Class	Purpose
ShapeError	ValueError	Raised when the shape of an exported model's output does not match the reference model's output shape.
AtolError	ValueError	Raised when the maximum absolute difference between reference and exported outputs exceeds the configured tolerance threshold.
OutputMatchError	ValueError	Raised when the exported model's output values do not match the reference in a general sense.
NumberOfInputsMatchError	ValueError	Raised when the number of inputs accepted by the exported model does not match the expected count.
NumberOfOutputsMatchError	ValueError	Raised when the number of outputs produced by the exported model does not match the reference model.
MinimumVersionError	ValueError	Raised when the installed version of a dependency (PyTorch, Transformers) does not meet the minimum version required for export.

Source:

class ShapeError(ValueError):
    pass

class AtolError(ValueError):
    pass

class OutputMatchError(ValueError):
    pass

class NumberOfInputsMatchError(ValueError):
    pass

class NumberOfOutputsMatchError(ValueError):
    pass

class MinimumVersionError(ValueError):
    pass

All error classes inherit from ValueError, allowing callers to catch them either individually or as a group via ValueError.

Import

from optimum.exporters.error_utils import (
    AtolError,
    OutputMatchError,
    ShapeError,
    NumberOfInputsMatchError,
    NumberOfOutputsMatchError,
    MinimumVersionError,
)
from optimum.exporters.base import ExporterConfig

Validation Flow

The typical validation flow during export:

1. The export function calls export_config.generate_dummy_inputs() to produce test inputs
2. The original model is run with these inputs to produce reference outputs
3. The exported model is loaded and run with the same inputs to produce test outputs
4. For each output pair (ref, test):
   • Check that shapes match (raise ShapeError if not)
   • Compute max_diff = max(abs(ref - test))
   • Look up tolerance: first check ATOL_FOR_VALIDATION_FOR_EACH_OUTPUT[output_name], then fall back to ATOL_FOR_VALIDATION
   • If max_diff > atol, raise AtolError
5. Verify total output count matches (raise NumberOfOutputsMatchError if not)

Usage Example

from optimum.exporters.error_utils import AtolError, ShapeError

# During validation (typically called by the export pipeline)
try:
    validate_model_outputs(
        config=export_config,
        reference_model=original_model,
        onnx_model_path="model.onnx",
        onnx_named_outputs=["logits"],
        atol=export_config.ATOL_FOR_VALIDATION,
    )
    print("Validation passed!")
except AtolError as e:
    print(f"Numerical mismatch: {e}")
except ShapeError as e:
    print(f"Shape mismatch: {e}")

# Customizing tolerance for a specific model config
class MyModelOnnxConfig(OnnxConfig):
    ATOL_FOR_VALIDATION = 1e-4  # More permissive global tolerance
    ATOL_FOR_VALIDATION_FOR_EACH_OUTPUT = {
        "attentions": 1e-3,  # Extra permissive for attention weights
    }

Related Pages

• Implements: Principle:Huggingface_Optimum_Export_Validation
• Environment:Huggingface_Optimum_Python_Core_Dependencies
• Heuristic:Huggingface_Optimum_Version_Conditional_Behavior