Implementation:Pytorch Serve Install Dependencies Script

Overview

Install_Dependencies_Script is the cross-platform dependency installer for TorchServe that automates the setup of Java, Node.js, PyTorch, and pip packages across Linux, macOS (Darwin), and Windows. It uses a class hierarchy with a Common base class and platform-specific subclasses (Linux, Windows, Darwin) to handle OS-specific installation logic, with support for CUDA, ROCm, and Neuron GPU backends.

Knowledge Sources	Pytorch_Serve
Domains	Infrastructure, DevOps
Last Updated	2026-02-13 18:52 GMT

Description

The install_dependencies.py script (424 lines) is the primary environment bootstrap tool for TorchServe. It detects the host operating system and delegates installation to the appropriate platform-specific class, ensuring all required system packages and Python dependencies are installed with the correct GPU backend configuration.

Class Hierarchy

Class	Lines	Description
`Common`	L86-198	Base class providing shared installation methods for Java, Node.js, pip packages, and PyTorch wheel selection
`Linux`	L200-255	Linux-specific implementation using `apt-get` / `yum` package managers
`Windows`	L257-276	Windows-specific implementation using `choco` package manager
`Darwin`	L278-321	macOS-specific implementation using `brew` package manager

Key Responsibilities

Platform Detection: Automatically identifies the host OS and selects the correct installer subclass
Java Installation: Installs OpenJDK (required by TorchServe's Java-based frontend)
Node.js Installation: Installs Node.js and npm (required for Newman test runner in dev environments)
PyTorch Wheel Selection: Constructs the correct pip index URL for PyTorch based on the specified CUDA, ROCm, or Neuron backend
Pip Package Installation: Installs all Python dependencies from requirements/common.txt or requirements/developer.txt
GPU Backend Support: Handles CUDA versions (cu92 through cu121), ROCm versions (rocm60-rocm62), and AWS Neuron

Code Reference

Source Location

File	Lines	Repository
`ts_scripts/install_dependencies.py`	L1-424	pytorch/serve

Main Entry Point

def install_dependencies(cuda_version=None, rocm_version=None, nightly=False):
    """
    Main entry point for installing all TorchServe dependencies.

    Detects the host platform, instantiates the appropriate OS-specific
    installer class, and runs all installation steps.

    Parameters:
        cuda_version (str|None): CUDA version identifier, e.g. "cu121", "cu118".
        rocm_version (str|None): ROCm version identifier, e.g. "rocm60", "rocm62".
        nightly (bool): If True, install nightly PyTorch builds from the nightly index.

    Raises:
        ValueError: If both cuda_version and rocm_version are provided.
    """
    # Lines 323-351
    ...

Common Base Class

class Common:
    """
    Base class for cross-platform dependency installation.
    Lines 86-198.
    """

    def install_java(self):
        """Install OpenJDK for TorchServe frontend."""
        ...

    def install_nodejs(self):
        """Install Node.js and npm for Newman test runner."""
        ...

    def install_torch_packages(self, cuda_version, rocm_version, nightly):
        """
        Install PyTorch, torchvision, and torchaudio with the correct
        GPU backend from the appropriate pip index URL.
        """
        ...

    def install_python_packages(self, cuda_version, rocm_version,
                                 requirements_file_path, nightly):
        """
        Install all Python dependencies from the requirements file
        and PyTorch with the specified GPU backend.
        """
        ...

    def install_node_packages(self):
        """Install npm packages (newman, markdown-link-check)."""
        ...

    def install_wget(self):
        """Install wget utility."""
        ...

    def install_numactl(self):
        """Install numactl for NUMA-aware process binding."""
        ...

Platform Subclasses

class Linux(Common):
    """
    Linux-specific installer. Lines 200-255.
    Uses apt-get or yum depending on the distribution.
    """

    def install_java(self):
        """Install openjdk-17-jdk via apt-get."""
        ...

    def install_nodejs(self):
        """Install Node.js via NodeSource repository setup."""
        ...


class Windows(Common):
    """
    Windows-specific installer. Lines 257-276.
    Uses Chocolatey package manager.
    """

    def install_java(self):
        """Install OpenJDK via choco."""
        ...


class Darwin(Common):
    """
    macOS-specific installer. Lines 278-321.
    Uses Homebrew package manager.
    """

    def install_java(self):
        """Install OpenJDK via brew."""
        ...

    def install_nodejs(self):
        """Install Node.js via brew."""
        ...

Import

# Typically invoked as a standalone script:
# python ts_scripts/install_dependencies.py [OPTIONS]

# When imported:
from ts_scripts.install_dependencies import install_dependencies

I/O Contract

Direction	Type	Description
Input	CLI arguments	`--cuda` (version string), `--rocm` (version string), `--environment` (prod/dev), `--nightly_torch` (flag), `--neuronx` (flag), `--cpp` (flag), `--force` (flag), `--skip_torch_install` (flag)
Output	Side effects	Installed system packages (Java, Node.js), Python packages (PyTorch, pip dependencies), and GPU-specific wheels
Precondition	System state	Python 3.x installed, pip available, OS package manager available (apt/yum/brew/choco), GPU drivers installed at OS level (for CUDA/ROCm)
Postcondition	System state	All packages from requirements file installed, PyTorch installed with correct GPU backend, Java JDK available on PATH

CLI Arguments

Argument	Type	Description
`--cuda`	`str`	CUDA version (cu92, cu101, cu102, cu111, cu113, cu116, cu117, cu118, cu121)
`--rocm`	`str`	ROCm version (rocm60, rocm61, rocm62)
`--environment`	`str`	Environment type: `prod` (default) or `dev`
`--nightly_torch`	flag	Install nightly PyTorch builds
`--neuronx`	flag	Install for AWS Neuron (Inferentia/Trainium)
`--cpp`	flag	Install C++ dependencies for cpp backend
`--force`	flag	Force reinstallation of packages
`--skip_torch_install`	flag	Skip PyTorch installation (use existing)

Usage Examples

Example 1: Production environment with CUDA 12.1

# Install all dependencies for production with CUDA 12.1
python ts_scripts/install_dependencies.py --cuda cu121 --environment prod

Example 2: Development environment setup

# Dev environment adds wget, Node.js, npm packages, and numactl
python ts_scripts/install_dependencies.py --cuda cu121 --environment dev

When --environment dev is specified, the script additionally installs:

if args.environment == "dev":
    system.install_wget()
    system.install_nodejs()
    system.install_node_packages()  # newman, markdown-link-check
    system.install_numactl()

Example 3: Platform detection flow

# The install_dependencies function detects the platform automatically:
import platform

os_map = {
    "Linux": Linux,
    "Windows": Windows,
    "Darwin": Darwin,
}
system = os_map[platform.system()]()

# Then runs the installation steps:
system.install_java()
system.install_python_packages(cuda_version, rocm_version, req_path, nightly)

Example 4: ROCm and Neuron backends

# ROCm 6.2 environment for AMD GPUs
python ts_scripts/install_dependencies.py --rocm rocm62

# AWS Neuron for Inferentia/Trainium chips
python ts_scripts/install_dependencies.py --neuronx

# Nightly PyTorch builds with CUDA
python ts_scripts/install_dependencies.py --cuda cu121 --nightly_torch

Related Pages

Principle:Pytorch_Serve_Environment_Setup - The principle of environment setup and dependency management for ML serving
Implementation:Pytorch_Serve_Install_Dependencies - Companion page documenting the pip requirements files and vLLM installation
Implementation:Pytorch_Serve_Print_Env_Info - Diagnostics script that verifies the installed environment
Implementation:Pytorch_Serve_Api_Utils - API test framework that requires Node.js/Newman installed by this script

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment