Implementation:Triton inference server Server GenQaModels

Knowledge Sources	Triton Inference Server
Domains	Testing, Model_Generation
Last Updated	2026-02-13 17:00 GMT

Overview

Primary QA model generator that creates add/subtract arithmetic test models across all supported backends.

Description

The `gen_qa_models.py` script is the largest and most widely used model generator in the QA suite, producing simple arithmetic models (add, subtract) that serve as the foundation for the majority of Triton integration tests. It generates models for every supported backend -- TensorRT, ONNX Runtime, TensorFlow SavedModel and GraphDef, TorchScript, OpenVINO, and the Python backend -- covering all data types and various batching configurations. The generated models accept two input tensors, produce an add output and a subtract output, and are used by dozens of QA tests to validate inference correctness, batching, scheduling, model management, and protocol handling.

Usage

Run this script as the first step in setting up a QA model repository. It is called by nearly every CI test script to create the baseline arithmetic models that the tests exercise.

Code Reference

Source Location

• Repository: Triton Inference Server
• File: qa/common/gen_qa_models.py
• Lines: 1-2347

Signature

def create_onnx_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_tf_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_plan_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_openvino_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_libtorch_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_modelconfig(models_dir, model_name, max_batch, dtype, shape, backend): ...
def create_models(models_dir, dtype, shape, no_batch=True): ...

Import

# Typically run as a standalone script
python qa/common/gen_qa_models.py --models_dir /tmp/models

I/O Contract

Inputs

Name	Type	Required	Description
models_dir	string	Yes	Output directory for generated model repository
dtype	string	No	Data type for model tensors (e.g., TYPE_FP32, TYPE_INT32)
shape	list[int]	No	Tensor shape for model inputs and outputs
no_batch	bool	No	Whether to also generate no-batch model variants
backend	string	No	Generate models only for a specific backend

Outputs

Name	Type	Description
model_repository	directory	Complete model repository with per-backend model directories
config.pbtxt	file	Model configuration specifying inputs (INPUT0, INPUT1) and outputs (OUTPUT0, OUTPUT1)
model files	file	Backend-specific model files implementing add and subtract operations

Usage Examples

Generate All QA Models

python qa/common/gen_qa_models.py \
    --models_dir /tmp/qa_models

Generate Models for Specific Backend

python qa/common/gen_qa_models.py \
    --models_dir /tmp/onnx_models \
    --backend onnxruntime

Common CI Usage

export DATADIR="/data/inferenceserver/${REPO_VERSION}/qa_model_repository"
python qa/common/gen_qa_models.py --models_dir $DATADIR

Related Pages

• Environment:Triton_inference_server_Server_GPU_CUDA_Runtime