Environment:Huggingface Transformers PyTorch 24 CUDA

Knowledge Sources	Huggingface Transformers import_utils.py
Domains	Infrastructure, Deep_Learning, GPU
Last Updated	2026-02-13 20:00 GMT

Overview

PyTorch >= 2.4 with CUDA GPU support, the primary deep learning backend for Transformers model training and inference.

Description

Transformers v5.x mandates PyTorch >= 2.4.0 as the minimum version for all torch-based operations. The library explicitly checks the PyTorch version at import time and disables torch support if the version is below 2.4.0. For GPU-accelerated workflows, NVIDIA CUDA or AMD ROCm drivers must be available. The library also supports alternative accelerators including Intel XPU (PyTorch >= 2.6), Huawei NPU (via torch_npu), Cambricon MLU (via torch_mlu), and Apple MPS.

Usage

Required for all GPU-accelerated workflows: model training, fine-tuning, quantized inference, distributed training, and benchmarking. CPU-only inference via pipelines can work without CUDA, but training and quantization workflows require a GPU.

System Requirements

Category	Requirement	Notes
OS	Linux (recommended), macOS, Windows	Linux for production GPU workloads
Hardware	NVIDIA GPU with CUDA support	Minimum: compute capability 7.0+ (Volta or newer)
VRAM	Depends on model size	8GB minimum for small models; 24GB+ for 7B models; 80GB for 70B
CUDA	11.8 or 12.x	Must match PyTorch build
Driver	NVIDIA Driver >= 525	For CUDA 12.x support

Dependencies

System Packages

NVIDIA CUDA Toolkit 11.8+ or 12.x
NVIDIA cuDNN 8.6+
nvidia-smi (for GPU monitoring)

Python Packages

torch >= 2.4.0
accelerate >= 1.1.0 (recommended for multi-device)
torchvision (for vision tasks)
torchaudio (for audio tasks)

Credentials

No credentials required for PyTorch + CUDA setup.

Quick Install

# Install Transformers with PyTorch
pip install transformers[torch]

# Or install PyTorch separately first (for specific CUDA version)
pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install transformers accelerate

Code Evidence

PyTorch >= 2.4 minimum version enforcement from src/transformers/utils/import_utils.py:111-119:

@lru_cache
def is_torch_available() -> bool:
    try:
        is_available, torch_version = _is_package_available("torch", return_version=True)
        parsed_version = version.parse(torch_version)
        if is_available and parsed_version < version.parse("2.4.0"):
            logger.warning_once(f"Disabling PyTorch because PyTorch >= 2.4 is required but found {torch_version}")
        return is_available and version.parse(torch_version) >= version.parse("2.4.0")
    except packaging.version.InvalidVersion:
        return False

CUDA availability detection from src/transformers/utils/import_utils.py:171-176:

@lru_cache
def is_torch_cuda_available() -> bool:
    if is_torch_available():
        import torch
        return torch.cuda.is_available()
    return False

Accelerate minimum version from src/transformers/utils/import_utils.py:96:

ACCELERATE_MIN_VERSION = "1.1.0"

Common Errors

Error Message	Cause	Solution
`Disabling PyTorch because PyTorch >= 2.4 is required`	Outdated PyTorch version	`pip install torch>=2.4`
`torch.cuda.is_available() returns False`	CUDA not installed or no GPU detected	Install NVIDIA drivers and CUDA toolkit
`CUDA out of memory`	Insufficient GPU VRAM	Reduce batch size, enable gradient checkpointing, or use quantization
`ImportError: accelerate`	accelerate not installed	`pip install accelerate>=1.1.0`

Compatibility Notes

NVIDIA CUDA: Primary supported platform. Version detection via torch.version.cuda.
AMD ROCm: Supported via torch.version.hip. Some features may require different package versions (e.g., flash_attn >= 2.0.4 for ROCm vs >= 2.1.0 for CUDA).
Intel XPU: Requires PyTorch >= 2.6.0. Detection via torch.xpu.is_available().
Apple MPS: Supported for inference on macOS. Detection via torch.backends.mps.is_available().
Huawei NPU: Requires torch_npu package. Detection via torch.npu.is_available().
Google TPU: Requires PyTorch/XLA. Controlled by USE_TORCH_XLA environment variable.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment