Environment:Deepspeedai DeepSpeed Python Runtime Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Deep_Learning |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
Python 3.8+ runtime environment with PyTorch and core dependencies required for all DeepSpeed operations.
Description
This environment defines the base Python software stack required by DeepSpeed. It includes the core pip dependencies declared in `requirements/requirements.txt`, the PyTorch framework (with or without GPU support), and various optional dependency groups for specific features (inference, autotuning, sparse attention, Stable Diffusion, Triton kernels, DeepCompile). DeepSpeed uses a JIT compilation model on Linux, where C++/CUDA extension ops are compiled at runtime when first needed, requiring a working C++ compiler and ninja build system.
Usage
This is the base environment required for all DeepSpeed workflows. Every Implementation page in this wiki implicitly requires this environment. Install the core package for training workflows, and add optional extras as needed for inference, autotuning, or specialized features.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| Python | 3.8, 3.9, 3.10, 3.11, or 3.12 | Listed in setup.py classifiers |
| OS | Linux (primary), Windows (limited) | Linux uses JIT compilation; Windows pre-compiles ops |
| C++ Compiler | GCC / Visual C++ Build Tools | Required for JIT compilation of DeepSpeed ops |
| Disk | Sufficient for pip packages + JIT cache | JIT-compiled ops are cached in `~/.cache/torch_extensions/` |
Dependencies
Core Python Packages
- `torch` (any version with matching CUDA if using GPU)
- `einops`
- `hjson`
- `msgpack`
- `ninja`
- `numpy`
- `packaging` >= 20.0
- `psutil`
- `py-cpuinfo`
- `pydantic` >= 2.0.0
- `tqdm`
Optional: Inference
- `google`
- `lm-eval` == 0.3.0
- `protobuf`
- `qtorch`
- `safetensors`
- `sentencepiece`
- `transformers` >= 4.32.1
Optional: Autotuning
Install with `pip install deepspeed[autotuning]`.
Optional: Stable Diffusion
- `diffusers` >= 0.25.0
- `triton` >= 2.1.0
Optional: Triton Kernels
- `triton` == 2.1.0
Optional: Sparse Attention
- `triton` == 1.0.0
Optional: DeepCompile
- `scipy`
Optional: Development
- `accelerate`
- `pytest` >= 7.2.0, < 8.4.0
- `transformers` >= 4.51.3
- `tensorboard`
- `wandb`
- `comet_ml` >= 3.41.0
- `deepspeed-kernels` (Linux only)
Credentials
The following environment variables affect the build and runtime:
- `DS_BUILD_OPS`: Set to `1` to pre-compile all compatible ops at install time.
- `DS_BUILD_STRING`: Custom build identifier string for distribution builds.
- `DS_ENABLE_NINJA`: Enable ninja for building C++ extensions.
- `DS_BUILD_<OP_NAME>`: Per-op build control (e.g., `DS_BUILD_FUSED_ADAM=1`).
Quick Install
# Basic install (ops JIT-compiled on demand)
pip install deepspeed
# Install with inference extras
pip install deepspeed[inf]
# Install with all extras
pip install deepspeed[all]
# Install with pre-compiled ops
DS_BUILD_OPS=1 pip install deepspeed
# Verify installation
ds_report
Code Evidence
Core requirements from `requirements/requirements.txt`:
einops
hjson
msgpack
ninja
numpy
packaging>=20.0
psutil
py-cpuinfo
pydantic>=2.0.0
torch
tqdm
Python version classifiers from `setup.py:328-332`:
classifiers=[
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12'
],
JIT vs pre-compile decision from `setup.py:151-157`:
# Default to pre-install kernels to false so we rely on JIT on Linux, opposite on Windows.
BUILD_OP_PLATFORM = 1 if sys.platform == "win32" else 0
BUILD_OP_DEFAULT = int(get_env_if_set('DS_BUILD_OPS', BUILD_OP_PLATFORM))
if BUILD_OP_DEFAULT:
assert torch_available, "Unable to pre-compile ops without torch installed."
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `Unable to import torch, pre-compiling ops will be disabled` | PyTorch not installed | `pip install torch` before installing DeepSpeed |
| `Unable to pre-compile ops without torch installed` | DS_BUILD_OPS=1 but no torch | Install PyTorch first, then re-install DeepSpeed |
| `Skip pre-compile of incompatible <op_name>` | Op not compatible with current system | This is a warning; op will be JIT-compiled if needed at runtime |
| JIT compilation fails | Missing C++ compiler or CUDA toolkit | Install `gcc` and `cuda-toolkit`; ensure `ninja` is available |
Compatibility Notes
- Windows: Ops are pre-compiled at install time (not JIT). Requires Visual C++ Build Tools and administrator privileges for symlink creation during setup.
- ROCm: cupy package selection varies by ROCm version. ROCm > 5.0 cupy support may be unavailable.
- CPU-only: DeepSpeed can run in CPU-only mode with reduced functionality. Set `DS_ACCELERATOR=cpu` if no GPU is detected.
- pydantic: Requires pydantic >= 2.0.0. DeepSpeed uses Pydantic models for configuration validation (`DeepSpeedConfigModel`).