Implementation:Triton inference server Server L0 Infer Test
| Knowledge Sources | |
|---|---|
| Domains | Testing, Inference |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
QA test script for validating basic inference operations in the Triton Inference Server.
Description
This test orchestrates basic inference validation across multiple backends and model types. It exercises the core inference path by generating model repositories containing models for supported backends (TensorRT, TensorFlow, ONNX Runtime, PyTorch), starting the server, and running inference requests through both HTTP and gRPC endpoints. The script validates inference correctness by comparing output tensors against expected values for various data types (FP32, FP16, INT8, INT16, INT32, etc.), input shapes, and batch sizes. It also tests multi-model concurrent inference, optional inputs, output tensor subsetting, and ensures consistent results across repeated inferences. This serves as the foundational inference smoke test for the Triton server.
Usage
Run as part of the Triton QA test suite to validate basic inference. Typically executed in CI/CD pipelines within the Triton Docker container environment.
Code Reference
Source Location
- Repository: Triton Inference Server
- File: qa/L0_infer/test.sh
- Lines: 1-446
Signature
#!/bin/bash
# Key functions and variables
source ../common/util.sh
SERVER=/opt/tritonserver/bin/tritonserver
SERVER_ARGS="--model-repository=${MODELDIR}"
SERVER_LOG="./infer_test.server.log"
CLIENT_LOG="./infer_test.client.log"
INFER_TEST="./infer_test.py"
Import
# Sources common utilities
source ../common/util.sh
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| DATADIR | env var | No | Directory containing test data |
| TRITON_DIR | env var | No | Path to Triton server installation |
| MODEL_REPO | directory | Yes | Generated test model repository with models for multiple backends |
Outputs
| Name | Type | Description |
|---|---|---|
| exit code | int | 0 on success, 1 on failure |
| test logs | files | Server logs and test output in test directory |
Usage Examples
Running the Test
cd qa/L0_infer/
bash test.sh