Implementation:Triton inference server Server Ensemble Config Pbtxt

Field	Value
Implementation Name	Ensemble_Config_Pbtxt
Implements	Principle:Triton_inference_server_Server_Ensemble_Configuration
Domains	Model_Serving, Configuration, Pipeline_Architecture
Status	Active
Last Updated	2026-02-13 17:00 GMT

Overview

Concrete config.pbtxt creation procedure for ensemble models using platform: "ensemble" with ensemble_scheduling. This implementation covers the full configuration file format, directory structure requirements, and validation rules.

Description

The ensemble config.pbtxt is a protobuf text format file that defines the ensemble model's external interface and internal DAG routing. Unlike standard model configurations, ensemble configs use platform: "ensemble" instead of specifying a backend, and the version directory must be empty (no model files).

Key requirements:

• Platform literal — Must be exactly platform: "ensemble"; this is a reserved platform type
• Empty version directory — The model must have at least one version directory (e.g., 1/) but it must be empty
• Composing model co-location — All composing models referenced in the steps must exist in the same model repository
• Tensor name consistency — Ensemble tensor names in input_map/output_map values must match the ensemble-level input/output declarations or connect to other steps

Usage

This implementation is used when creating the configuration file for an ensemble model after all component models have been prepared and their tensor interfaces are known.

Code Reference

Source Location

• docs/user_guide/ensemble_models.md:L60-123 — Complete ensemble configuration documentation
• qa/common/gen_ensemble_model_utils.py:L768-847 — create_ensemble_modelconfig function
• qa/common/gen_ensemble_model_utils.py:L1094-1198 — create_general_modelconfig function

Signature

name: "ensemble_add_sub"
platform: "ensemble"
max_batch_size: 8

input [
  { name: "INPUT0", data_type: TYPE_FP32, dims: [ 16 ] },
  { name: "INPUT1", data_type: TYPE_FP32, dims: [ 16 ] }
]
output [
  { name: "OUTPUT0", data_type: TYPE_FP32, dims: [ 16 ] },
  { name: "OUTPUT1", data_type: TYPE_FP32, dims: [ 16 ] }
]

ensemble_scheduling {
  step [
    {
      model_name: "add_sub_model"
      model_version: -1
      input_map {
        key: "INPUT0"
        value: "INPUT0"
      }
      input_map {
        key: "INPUT1"
        value: "INPUT1"
      }
      output_map {
        key: "OUTPUT0"
        value: "OUTPUT0"
      }
      output_map {
        key: "OUTPUT1"
        value: "OUTPUT1"
      }
    }
  ]
}

Import

No import required. This is a protobuf text format configuration file placed at model_repository/ensemble_name/config.pbtxt.

Key Requirements

Requirement	Details
Platform	Must be "ensemble" (literal string)
Version directory	Must exist and be empty (e.g., mkdir -p ensemble_name/1/)
Composing models	Must exist in the same model repository
Tensor types	Must match across connected steps (ensemble tensor → composing model input)
Batch size	Ensemble max_batch_size must not exceed any composing model's max_batch_size

I/O Contract

Inputs

Input	Type	Description
Tensor routing design	DAG specification	The desired topology (which models connect to which)
Composing model names	string	Names of component models already in the model repository
Composing model specs	tensor metadata	Input/output tensor names, data types, and shapes for each composing model

Outputs

Output	Type	Description
config.pbtxt	file	Ensemble configuration file placed in the model repository at ensemble_name/config.pbtxt
Empty version directory	directory	Empty directory at ensemble_name/1/ (or other version number)

Usage Examples

Directory structure for an ensemble model:

model_repository/
  ensemble_add_sub/
    config.pbtxt          # Ensemble configuration
    1/                     # Empty version directory (no model files)
  add_sub_model/
    config.pbtxt          # Composing model configuration
    1/
      model.onnx          # Actual model file

Creating the directory structure:

# Create ensemble model directory with empty version folder
mkdir -p model_repository/ensemble_add_sub/1/

# Create the config.pbtxt (content as shown in Signature above)
# Place it at model_repository/ensemble_add_sub/config.pbtxt

Multi-step ensemble config (preprocess → classify → postprocess):

name: "image_pipeline"
platform: "ensemble"
max_batch_size: 4

input [
  { name: "RAW_IMAGE", data_type: TYPE_UINT8, dims: [ 224, 224, 3 ] }
]
output [
  { name: "LABEL", data_type: TYPE_STRING, dims: [ 1 ] },
  { name: "CONFIDENCE", data_type: TYPE_FP32, dims: [ 1 ] }
]

ensemble_scheduling {
  step [
    {
      model_name: "preprocess"
      model_version: -1
      input_map { key: "RAW_IMAGE"  value: "RAW_IMAGE" }
      output_map { key: "PROCESSED"  value: "preprocessed_tensor" }
    },
    {
      model_name: "classifier"
      model_version: -1
      input_map { key: "INPUT"  value: "preprocessed_tensor" }
      output_map { key: "SCORES"  value: "raw_scores" }
    },
    {
      model_name: "postprocess"
      model_version: -1
      input_map { key: "SCORES"  value: "raw_scores" }
      output_map { key: "LABEL"  value: "LABEL" }
      output_map { key: "CONFIDENCE"  value: "CONFIDENCE" }
    }
  ]
}

Related Pages

• implements::Principle:Triton_inference_server_Server_Ensemble_Configuration