Implementation:Triton inference server Server GenQaTrtPluginModels
| Knowledge Sources | |
|---|---|
| Domains | Testing, Model_Generation |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
Generates TensorRT models that use custom TensorRT plugins to test Triton's plugin loading and execution support.
Description
The `gen_qa_trt_plugin_models.py` script builds TensorRT engine plans that incorporate custom plugins, verifying that Triton can correctly load plugin libraries and execute models that depend on them. It registers custom plugin creators, builds TensorRT networks using plugin layers, and serializes the resulting engine plans. These models are used by QA tests that validate Triton's `--backend-config` plugin path settings and ensure custom operations execute correctly during inference.
Usage
Run this script to generate TensorRT models with custom plugins before testing plugin support. Requires TensorRT development libraries and the custom plugin shared library to be available.
Code Reference
Source Location
- Repository: Triton Inference Server
- File: qa/common/gen_qa_trt_plugin_models.py
- Lines: 1-309
Signature
def create_plan_modelfile_with_plugin(models_dir, model_version, max_batch, dtype, shape): ...
def create_modelconfig(models_dir, model_name, max_batch, dtype, shape): ...
def create_models(models_dir): ...
Import
# Typically run as a standalone script
python qa/common/gen_qa_trt_plugin_models.py --models_dir /tmp/models
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| models_dir | string | Yes | Output directory for generated model repository |
| plugin_lib | string | No | Path to the custom TensorRT plugin shared library (.so) |
Outputs
| Name | Type | Description |
|---|---|---|
| model_repository | directory | Model directories with plugin-enabled TensorRT plans |
| config.pbtxt | file | Model configuration for the plugin-based model |
| model.plan | file | TensorRT engine plan serialized with custom plugin layers |
Usage Examples
Generate TRT Plugin Models
python qa/common/gen_qa_trt_plugin_models.py \
--models_dir /tmp/trt_plugin_models
Run with Plugin Library
MODELS_DIR="/tmp/trt_plugin_models"
python qa/common/gen_qa_trt_plugin_models.py --models_dir $MODELS_DIR
tritonserver --model-repository=$MODELS_DIR \
--backend-config=tensorrt,plugins=/path/to/libcustomplugin.so &