Implementation:Triton inference server Server GenQaReshapeModels
| Knowledge Sources | |
|---|---|
| Domains | Testing, Model_Generation |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
Generates test models with reshape operations to validate Triton's tensor reshape handling between client and model tensor shapes.
Description
The `gen_qa_reshape_models.py` script creates models where the client-facing tensor shapes differ from the internal model tensor shapes, requiring Triton to apply reshape transformations. It generates models with `reshape` directives in the model configuration, producing models across TensorRT, ONNX, TensorFlow, and TorchScript backends. These models test Triton's ability to correctly reshape tensors before passing them to the model and reshape outputs back to the client-expected shape, including scenarios with variable-size dimensions and batch dimension handling.
Usage
Run this script to create reshape test models before executing QA tests that validate Triton's tensor reshape feature. Used in CI to test both fixed and dynamic reshape configurations.
Code Reference
Source Location
- Repository: Triton Inference Server
- File: qa/common/gen_qa_reshape_models.py
- Lines: 1-1273
Signature
def create_onnx_modelfile(models_dir, model_version, max_batch, dtype, input_shape, output_shape): ...
def create_tf_modelfile(models_dir, model_version, max_batch, dtype, input_shape, output_shape): ...
def create_plan_modelfile(models_dir, model_version, max_batch, dtype, input_shape, output_shape): ...
def create_libtorch_modelfile(models_dir, model_version, max_batch, dtype, input_shape, output_shape): ...
def create_modelconfig(models_dir, model_name, max_batch, dtype, input_shape, output_shape, reshape_pairs): ...
def create_models(models_dir, dtype): ...
Import
# Typically run as a standalone script
python qa/common/gen_qa_reshape_models.py --models_dir /tmp/models
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| models_dir | string | Yes | Output directory for generated model repository |
| dtype | string | No | Data type for model tensors |
| input_shape | list[int] | No | Client-facing input tensor shape |
| output_shape | list[int] | No | Client-facing output tensor shape |
| reshape_pairs | list[tuple] | No | Pairs of (client_shape, model_shape) for reshape configuration |
Outputs
| Name | Type | Description |
|---|---|---|
| model_repository | directory | Model directories with reshape-configured model files |
| config.pbtxt | file | Model configuration with reshape directives on inputs and outputs |
| model files | file | Backend-specific model files operating on reshaped tensor dimensions |
Usage Examples
Generate Reshape Models
python qa/common/gen_qa_reshape_models.py \
--models_dir /tmp/reshape_models
CI Test Script Usage
MODELS_DIR="${DATADIR}/qa_reshape_model_repository"
python qa/common/gen_qa_reshape_models.py --models_dir $MODELS_DIR
tritonserver --model-repository=$MODELS_DIR --strict-model-config=true &