Implementation:Pytorch Serve Install Dependencies

Field	Value
Page Type	Implementation
Implementation Type	External Tool Doc
Domains	Infrastructure, DevOps
Knowledge Sources	TorchServe
Workflow	LLM_Deployment_vLLM
Last Updated	2026-02-13 00:00 GMT

Overview

This page documents the concrete installation procedure for TorchServe and its dependencies, including the vLLM inference engine for LLM deployment. The primary entry points are the requirements/common.txt pip requirements file and the ts_scripts/install_dependencies.py installation script, which together automate the full environment setup across Linux, macOS, and Windows platforms.

Description

The TorchServe dependency installation consists of two coordinated mechanisms:

1. Pip Requirements File (requirements/common.txt) defines the core Python packages:

psutil==5.9.8
requests==2.32.0
captum==0.6.0
packaging==23.2
pynvml==11.5.0
pyyaml==6.0.1
ninja==1.11.1.1
setuptools

2. Installation Script (ts_scripts/install_dependencies.py) orchestrates the full setup including system packages, Java, Node.js, and PyTorch with the correct GPU backend.

3. vLLM Engine is installed separately via its own requirements file at examples/large_models/vllm/requirements.txt, which pins vLLM version 0.6.1.post2.

Usage

Basic Installation

# Install TorchServe core packages
pip install torchserve torch-model-archiver torch-workflow-archiver

# Install common dependencies
pip install -r requirements/common.txt

# Install vLLM for LLM serving
pip install vllm==0.6.1.post2

Full Environment Setup with Script

# Production environment with CUDA 12.1
python ts_scripts/install_dependencies.py --cuda cu121 --environment prod

# Development environment with CUDA 12.1
python ts_scripts/install_dependencies.py --cuda cu121 --environment dev

# ROCm 6.2 environment
python ts_scripts/install_dependencies.py --rocm rocm62

# Nightly PyTorch builds
python ts_scripts/install_dependencies.py --cuda cu121 --nightly_torch

Code Reference

Source Location

File	Lines	Purpose
requirements/common.txt	L1-8	Core Python dependency manifest
ts_scripts/install_dependencies.py	L323-347	Main install_dependencies() function
ts_scripts/install_dependencies.py	L145-176	install_python_packages() method
ts_scripts/install_dependencies.py	L97-143	install_torch_packages() method
examples/large_models/vllm/requirements.txt	L1	vLLM version pin

Signature

def install_dependencies(cuda_version=None, rocm_version=None, nightly=False):
    """
    Main entry point for installing all TorchServe dependencies.

    Parameters:
        cuda_version (str|None): CUDA version identifier, e.g. "cu121", "cu118".
            Choices: cu92, cu101, cu102, cu111, cu113, cu116, cu117, cu118, cu121
        rocm_version (str|None): ROCm version identifier, e.g. "rocm60", "rocm61", "rocm62".
        nightly (bool): If True, install nightly PyTorch builds from the nightly index.

    Raises:
        ValueError: If both cuda_version and rocm_version are provided.
    """

def install_python_packages(self, cuda_version, rocm_version, requirements_file_path, nightly):
    """
    Install PyTorch and Python dependencies from the appropriate requirements file.

    Parameters:
        cuda_version (str|None): CUDA version for PyTorch wheel selection.
        rocm_version (str|None): ROCm version for PyTorch wheel selection.
        requirements_file_path (str): Path to the requirements file (common.txt or developer.txt).
        nightly (bool): Whether to install nightly PyTorch builds.
    """

Import

# The install script is run as a standalone script, not imported as a module
# Usage: python ts_scripts/install_dependencies.py [OPTIONS]

# For pip-based installation of TorchServe itself:
# pip install torchserve torch-model-archiver vllm

I/O Contract

Direction	Type	Description
Input	CLI arguments	--cuda (version string), --rocm (version string), --environment (prod/dev), --nightly_torch (flag), --neuronx (flag), --cpp (flag), --force (flag), --skip_torch_install (flag)
Output	Side effects	Installed Python packages, system packages (Java JDK, Node.js), and GPU-specific PyTorch wheels
Precondition	System state	Python 3.x installed, pip available, GPU drivers installed at OS level (for CUDA/ROCm)
Postcondition	System state	All packages from requirements/common.txt installed, PyTorch installed with correct GPU backend, Java JDK available

Usage Examples

Example 1: Install for Production with CUDA 12.1

# This installs Java, PyTorch with CUDA 12.1, and common.txt dependencies
python ts_scripts/install_dependencies.py --cuda cu121

# Then install vLLM separately
pip install vllm==0.6.1.post2

The script internally performs:

# 1. Detects platform (Linux/Windows/Darwin)
os_map = {"Linux": Linux, "Windows": Windows, "Darwin": Darwin}
system = os_map[platform.system()]()

# 2. Installs Java
system.install_java()

# 3. Selects requirements file based on environment
requirements_file = "common.txt"  # for prod
requirements_file_path = os.path.join("requirements", requirements_file)

# 4. Installs PyTorch with correct CUDA version and common dependencies
system.install_python_packages(cuda_version, rocm_version, requirements_file_path, nightly)

Example 2: Development Environment Setup

# Dev environment adds wget, Node.js, npm packages, and numactl
python ts_scripts/install_dependencies.py --cuda cu121 --environment dev

When --environment dev is specified, the script additionally installs:

if args.environment == "dev":
    system.install_wget()
    system.install_nodejs()
    system.install_node_packages()  # newman, markdown-link-check
    system.install_numactl()

Example 3: Key Dependencies Explained

Package	Version	Role in TorchServe
psutil	5.9.8	System metrics collection (CPU, memory, disk)
requests	2.32.0	HTTP client for management API calls and health checks
captum	0.6.0	Model interpretability and explanations
packaging	23.2	Version comparison for PyTorch compatibility checks
pynvml	11.5.0	NVIDIA GPU monitoring (utilization, memory, temperature)
pyyaml	6.0.1	Parsing model-config.yaml and config.properties
ninja	1.11.1.1	Build system for JIT-compiled custom CUDA kernels
setuptools	latest	Package metadata and installation utilities
vllm	0.6.1.post2	High-throughput LLM inference engine with PagedAttention

Related Pages

• Principle:Pytorch_Serve_Environment_Setup -- the theoretical basis for environment setup and dependency management in ML serving
• Environment:Pytorch_Serve_Python_PyTorch_Runtime - Base Python/PyTorch/Java runtime to install