Implementation:Deepspeedai DeepSpeed CPU Accelerator

Knowledge Sources	DeepSpeed
Domains	Accelerator, CPU Backend
Last Updated	2026-02-09 00:00 GMT

Overview

Intel CPU backend implementation enabling DeepSpeed execution on CPU-only systems without GPU hardware.

Description

The CPU_Accelerator class implements the DeepSpeedAccelerator interface for Intel CPU execution. It reports as a synchronized device where operations execute sequentially without async kernel launches. Memory tracking uses psutil to monitor RSS (Resident Set Size) instead of device memory APIs. Device count is determined by NUMA node count, with special handling for HBM in flat mode. The communication backend defaults to ccl (via oneccl_bindings_for_pytorch) or falls back to gloo if OneCCL is unavailable. Streams, events, and graph operations return no-ops since CPUs lack these GPU-specific constructs.

Usage

Use when running DeepSpeed on CPU-only environments, for development/testing, or when GPU hardware is unavailable. Automatically selected as fallback when no GPU or other accelerator is detected.

Code Reference

Source Location

• Repository: DeepSpeed
• File: accelerator/cpu_accelerator.py

Signature

class CPU_Accelerator(DeepSpeedAccelerator):
    def __init__(self):
        self._name = 'cpu'
        self._compile_backend = "inductor"
        if oneccl_imported_p:
            self._communication_backend_name = 'ccl'
        else:
            self._communication_backend_name = 'gloo'

    def is_synchronized_device(self):
        return True

    def device_name(self, device_index=None):
        return 'cpu'

    def device_count(self):
        # Returns NUMA node count

    def get_rss(self):
        import psutil
        return psutil.Process().memory_info().rss

    def memory_allocated(self, device_index=None):
        return self.get_rss()

    def get_op_builder(self, class_name):
        # Returns CCLCommBuilder, ShareMemCommBuilder,
        # FusedAdamBuilder, CPUAdamBuilder, AsyncIOBuilder

Import

from deepspeed.accelerator.cpu_accelerator import CPU_Accelerator

I/O Contract

Inputs

Name	Type	Required	Description
device_index	int	Optional	Ignored for CPU (single device)
seed	int	Required	Random seed for torch.manual_seed

Outputs

Name	Type	Description
device_name	str	Always 'cpu'
device_count	int	Number of NUMA nodes with cores
memory_bytes	int	RSS memory from psutil
communication_backend	str	'ccl' or 'gloo'

Usage Examples

# Explicitly set CPU accelerator
import os
os.environ['DS_ACCELERATOR'] = 'cpu'

from deepspeed.accelerator import get_accelerator
accelerator = get_accelerator()

print(f"Device: {accelerator.device_name()}")  # 'cpu'
print(f"Backend: {accelerator.communication_backend_name()}")  # 'ccl' or 'gloo'
print(f"Memory: {accelerator.memory_allocated()}")  # RSS in bytes

# CPU supports BF16 but FP16 support depends on MKL-DNN
print(f"BF16: {accelerator.is_bf16_supported()}")  # True
print(f"FP16: {accelerator.is_fp16_supported()}")  # Varies

# Get CPU-specific op builders
ccl_builder = accelerator.get_op_builder('CCLCommBuilder')
adam_builder = accelerator.get_op_builder('CPUAdamBuilder')

Related Pages

• Abstract Accelerator - Base interface
• Real Accelerator - Accelerator detection and selection
• CUDA Accelerator - GPU alternative