Implementation:Triton inference server Server GenQaDynaSequenceImplicitModels
| Knowledge Sources | |
|---|---|
| Domains | Testing, Model_Generation |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
Generates test models for dynamic sequence batching with implicit state management across multiple backends.
Description
The `gen_qa_dyna_sequence_implicit_models.py` script creates model repository artifacts that combine dynamic sequence batching with implicit state, where the model internally tracks sequence state rather than receiving it as an explicit input. It generates models for backends including TensorRT, ONNX Runtime, TensorFlow, and the Python backend, each configured with correlation ID handling and implicit state tensors. These models are consumed by QA tests that validate Triton's ability to manage dynamic sequences while the model maintains its own accumulated state.
Usage
Run this script before executing dynamic sequence implicit state QA tests to populate the model repository with the required test models. Typically invoked from CI test scripts or manually during test development.
Code Reference
Source Location
- Repository: Triton Inference Server
- File: qa/common/gen_qa_dyna_sequence_implicit_models.py
- Lines: 1-867
Signature
def create_onnx_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_tf_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_plan_modelfile(models_dir, model_version, max_batch, dtype, shape): ...
def create_modelconfig(models_dir, model_name, max_batch, dtype, shape): ...
def create_models(models_dir, dtype, shape, no_batch=True): ...
Import
# Typically run as a standalone script
python qa/common/gen_qa_dyna_sequence_implicit_models.py --models_dir /tmp/models
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| models_dir | string | Yes | Output directory for generated model repository |
| dtype | string | No | Data type for model tensors (e.g., int32, fp32) |
| shape | list[int] | No | Tensor shape for model inputs/outputs |
| no_batch | bool | No | Whether to also generate no-batch variants |
Outputs
| Name | Type | Description |
|---|---|---|
| model_repository | directory | Model directories with versioned model files and config.pbtxt |
| config.pbtxt | file | Model configuration with dynamic sequence batcher and implicit state settings |
| model files | file | Backend-specific model files (ONNX, SavedModel, TensorRT plan) |
Usage Examples
Generate All Dynamic Sequence Implicit Models
python qa/common/gen_qa_dyna_sequence_implicit_models.py \
--models_dir /tmp/dyna_sequence_implicit_models
Use in a CI Test Script
MODELS_DIR=$(mktemp -d)
python qa/common/gen_qa_dyna_sequence_implicit_models.py --models_dir $MODELS_DIR
tritonserver --model-repository=$MODELS_DIR &