Implementation:Triton inference server Server Backend Selection

Field	Value
Page Type	Implementation
Title	Backend_Selection
Namespace	Triton_inference_server_Server
Workflow	Custom_Container_Build
Domains	MLOps, Container_Build, Configuration
Last Updated	2026-02-13 17:00 GMT

Overview

Concrete process for selecting Triton backends and features using build.py's enable flags.

Description

Backend selection is performed through CLI flags passed to either build.py (source build) or compose.py (compose build). The flags control which inference backends, network endpoints, and cloud filesystem integrations are compiled or extracted into the final container image.

The --enable-all flag is a convenience shortcut that activates all supported backends, endpoints, and filesystems. For fine-grained control, operators specify individual --backend, --endpoint, and --filesystem flags.

The available backends span the major ML framework ecosystems:

ensemble -- Always included as a core backend; enables model ensembles (pipelines of models)
identity, square, repeat -- Test backends for development and CI
onnxruntime -- ONNX Runtime for ONNX model format
python -- Python backend for custom Python model code
dali -- NVIDIA DALI for data preprocessing pipelines
pytorch -- LibTorch backend for PyTorch models
openvino -- Intel OpenVINO for optimized CPU inference
fil -- RAPIDS Forest Inference Library for tree-based models
tensorrt -- NVIDIA TensorRT for optimized GPU inference

Usage

Selecting Specific Backends

# Select only TensorRT and Python backends with HTTP/gRPC endpoints
python3 build.py \
  --backend tensorrt \
  --backend python \
  --endpoint http \
  --endpoint grpc \
  --enable-gpu

# Select ONNX Runtime with S3 filesystem support
python3 build.py \
  --backend onnxruntime \
  --endpoint http \
  --endpoint grpc \
  --filesystem s3 \
  --enable-gpu

Enabling All Backends

# Enable everything for a full-featured build
python3 build.py --enable-all

Specifying Backend Versions

# Pin a specific backend repository tag
python3 build.py \
  --backend onnxruntime:r24.12 \
  --backend python:r24.12 \
  --endpoint http \
  --endpoint grpc

Code Reference

Source Location

File	Lines	Description
`build.py`	L2353-2431	`enable_all()` function that sets all backend, endpoint, and filesystem flags
`build.py`	L73-83	`DEFAULT_TRITON_VERSION_MAP` defining default version mappings
`docs/customization_guide/build.md`	L29-66	Documentation of available backends and flags

Signature

CLI flags:
  --enable-all                         Enable all backends, endpoints, and filesystems
  --backend <name>[:<repo-tag>]        Enable a specific backend (repeatable)
  --endpoint <name>                    Enable a network endpoint: grpc, http, sagemaker, vertex-ai
  --filesystem <name>                  Enable a cloud filesystem: gcs, s3, azure_storage
  --enable-gpu                         Enable GPU support (CUDA, TensorRT)

Import

# Internal constants used by build.py for backend selection
CORE_BACKENDS = ["ensemble"]  # Always included regardless of flags
DEFAULT_TRITON_VERSION_MAP  # Maps component names to default version tags (build.py:L73-83)

I/O Contract

Inputs

Input	Type	Description
User requirements	CLI flags	Backend, endpoint, and filesystem selections via command-line arguments
Target platform	CLI flag	GPU enablement (`--enable-gpu`) and build type (`--build-type`)
Version overrides	CLI flag	Optional backend version pinning via `--backend name:tag` syntax

Outputs

Output	Type	Description
Backend list	Internal list	Set of backend names to compile or extract
Endpoint list	Internal list	Set of network endpoints to enable
Filesystem list	Internal list	Set of cloud storage backends to enable
Build path decision	Internal flag	Whether to use compose (binary extraction) or source (compilation)

Usage Examples

Example 1: Minimal CPU-only build for ONNX models

python3 build.py \
  --backend onnxruntime \
  --endpoint http \
  --endpoint grpc

This produces a container with only the ONNX Runtime backend, HTTP and gRPC endpoints, no GPU support, and no cloud filesystem integration. The resulting image is significantly smaller than a full build.

Example 2: Cloud-native GPU build for AWS

python3 build.py \
  --backend tensorrt \
  --backend python \
  --backend onnxruntime \
  --endpoint http \
  --endpoint grpc \
  --filesystem s3 \
  --enable-gpu

This produces a GPU-enabled container with three backends and S3 filesystem support for loading models directly from Amazon S3 buckets.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment