Principle:Huggingface Optimum Backend Availability Detection

Overview

Mechanism for detecting which hardware acceleration backends are available in the current Python environment.

Description

Before routing inference to an accelerated backend, the system must detect which backends are actually installed. This includes ONNX Runtime (with 17 distribution variants like onnxruntime, onnxruntime-gpu, onnxruntime-directml, onnxruntime-rocm, onnxruntime-training, and others), OpenVINO, and Intel Extension for PyTorch (IPEX). Detection uses Python's importlib to check package availability at import time, caching results in module-level variables for efficiency.

The detection mechanism works in two phases:

Spec check: Uses importlib.util.find_spec() to verify the package is importable
Distribution check: Uses importlib.metadata.version() to verify the package is properly installed (not just an importable directory) and retrieves its version

This two-phase approach prevents false positives from partially installed or corrupted packages.

Usage

Use when initializing accelerated inference pipelines to determine which backends can be used. The availability functions are the first step in the Accelerated Inference Pipeline workflow -- they gate whether a particular backend can be selected.

from optimum.utils import is_onnxruntime_available, is_openvino_available, is_ipex_available

if is_openvino_available():
    # Use OpenVINO backend
    pass
elif is_onnxruntime_available():
    # Use ONNX Runtime backend
    pass
elif is_ipex_available():
    # Use IPEX backend
    pass

Theoretical Basis

Capability detection pattern using importlib.metadata and importlib.util. Results are cached at module load time in module-level boolean variables (e.g., _onnxruntime_available, _openvino_available, _ipex_available) to avoid repeated import checks on every call.

Priority order for auto-selection: When no accelerator is explicitly specified, the system selects the best available backend in this priority order:

Priority	Backend	Condition
1 (Highest)	OpenVINO (`ov`)	`optimum-intel` and `openvino` installed
2	ONNX Runtime (`ort`)	`optimum-onnx` and `onnxruntime` installed
3 (Lowest)	IPEX (`ipex`)	`optimum-intel` and `intel_extension_for_pytorch` installed

This priority order reflects the typical performance advantage and breadth of hardware support of each backend.

ONNX Runtime Distribution Variants

The ONNX Runtime availability check is notable for supporting 17 distribution names, since ONNX Runtime is published under different package names for different hardware targets:

Distribution Name	Target Hardware
`onnxruntime`	CPU (default)
`onnxruntime-gpu`	NVIDIA GPU (CUDA)
`onnxruntime-rocm`	AMD GPU (ROCm)
`onnxruntime-training`	Training support
`onnxruntime-training-rocm`	Training on AMD GPU
`onnxruntime-training-cpu`	Training on CPU
`onnxruntime-openvino`	OpenVINO execution provider
`onnxruntime-vitisai`	Xilinx Vitis AI
`onnxruntime-armnn`	ARM Neural Networks
`onnxruntime-cann`	Huawei CANN
`onnxruntime-dnnl`	Intel DNNL
`onnxruntime-acl`	ARM Compute Library
`onnxruntime-tvm`	Apache TVM
`onnxruntime-qnn`	Qualcomm QNN
`onnxruntime-migraphx`	AMD MIGraphX
`ort-migraphx-nightly`	AMD MIGraphX (nightly)
`ort-rocm-nightly`	AMD ROCm (nightly)

Metadata

Field	Value
Source Repository	optimum
Domains	Inference, Runtime_Detection

Connections

Implementation:Huggingface_Optimum_Is_Backend_Available

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment