Implementation:Triton inference server Server L0 Storage S3 Local Test

L0 Storage S3 Local Test

Source File: qa/L0_storage_S3_local/test.sh
Language: Bash (386 lines)
Domains: Testing, Cloud_Storage

Purpose

This QA test shell script validates Triton Inference Server's ability to load and serve models from a local S3-compatible storage backend. It provisions a local MinIO server to simulate Amazon S3 and exercises model repository operations over the S3 protocol, including hostname/IP access, polling-based model control, autocomplete configuration, multi-repository setups, access credential validation, and HTTP/2 advertisement behavior.

Signature

#!/bin/bash
# Primary entry point: test.sh [REPO_VERSION]
# Environment variables:
#   NVIDIA_TRITON_SERVER_VERSION - Repository version
#   BACKENDS - Space-separated list of backends (default: "onnx libtorch plan")
#   CUDA_VISIBLE_DEVICES - GPU device selection (set to 0)
#
# Key functions:
#   run_unit_tests()        - Executes infer_test.py and validates results
#   setup_model_repo()      - Copies model artifacts into a local directory
#   load_models()           - Loads models via HTTP model control API

Key Components

MinIO Server Setup

The script downloads, installs, and starts a local MinIO server to provide an S3-compatible endpoint on 127.0.0.1:4572. Credentials are configured via MINIO_ACCESS_KEY and MINIO_SECRET_KEY environment variables and mapped to AWS credential variables for the awscli-local tool.

export MINIO_ACCESS_KEY="minio"
export MINIO_SECRET_KEY="miniostorage"
export MINIO_CI_CD=true
/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES &
MINIO_PID=$!

export AWS_ACCESS_KEY_ID=minio
export AWS_SECRET_ACCESS_KEY=miniostorage

Hostname and IP Address Tests

The first test block iterates over both 127.0.0.1 and localhost to confirm Triton can connect to S3 using either a hostname or an IP address. The server is started in explicit model control mode, models are loaded via the HTTP API, and inference tests are run. A model with special characters in its name (Model_repo-1.0) is also tested.

for HOST in "127.0.0.1" "localhost"; do
    SERVER_ARGS="--model-repository=s3://$HOST:4572/demo-bucket1.0 --model-control-mode=explicit"
    run_server
    load_models
    run_unit_tests
    # Test model name with allowed special characters
    code=`curl -s -w %{http_code} -X POST localhost:8000/v2/repository/models/${DUMMY_MODEL}/load`
done

Polling Mode Tests

A separate test block verifies poll-based model control. After syncing a new model version (version 4) to S3, the script waits for the server to detect the change and validates that version 3 is unloaded while version 4 becomes READY.

SERVER_ARGS="--model-repository=s3://localhost:4572/demo-bucket1.0 --model-control-mode=poll"
# Add a new version and sync to S3
cp -r models/libtorch_float32_float32_float32/1 models/libtorch_float32_float32_float32/4
awslocal $ENDPOINT_FLAG s3 sync models s3://demo-bucket1.0
sleep 20
# Verify version transitions

Autocomplete Tests

Tests that Triton can automatically complete model configuration when --strict-model-config=false is used. The script strips input/output definitions from ONNX model configs and expects Triton to fill them in automatically when loading from S3.

Multiple Model Repository Tests

Validates that Triton can load models from multiple S3 buckets simultaneously by splitting backends across two separate buckets and providing multiple --model-repository arguments.

Access Decline Test

Verifies that Triton produces a clear error message when given invalid S3 credentials and refuses to start the server, checking for the specific error message about S3 filesystem client creation failure.

export AWS_SECRET_ACCESS_KEY="[Invalid]"
SERVER_ARGS="--model-repository=s3://localhost:4572/${BUCKET_NAME}1 --exit-timeout-secs=120"
run_server
# Expect server to NOT start
EXPECTED_MSG="Unable to create S3 filesystem client. Check account credentials."

HTTP/2 Advertisement Test

Confirms that the S3 client does not advertise HTTP/2 support by starting a mock S3 service and checking the connection behavior.

Test Flow

1. Install and start local MinIO server
2. Create S3 bucket and sync model repository
3. Run hostname/IP inference tests with explicit model control
4. Run polling mode test with version change detection
5. Run autocomplete test with stripped model configs
6. Run multiple model repository test across two S3 buckets
7. Run access decline test with invalid credentials
8. Run HTTP/2 advertisement test with mock S3 service
9. Aggregate and report pass/fail results

Dependencies

• minio - Local S3-compatible object storage server
• awscli-local (v0.07) - AWS CLI wrapper for local endpoints
• ../common/util.sh - Common test utility functions
• ../common/infer_test.py - Inference validation test script
• mock_s3_service.py - Mock S3 service for HTTP/2 testing