Implementation:Microsoft Onnxruntime OrtTensorRTProviderOptions

Knowledge Sources	Description
Source File	java/src/main/java/ai/onnxruntime/providers/OrtTensorRTProviderOptions.java
Repository	Microsoft/onnxruntime

Domains

• Machine Learning Runtime
• GPU Acceleration
• TensorRT Configuration

Overview

OrtTensorRTProviderOptions is a final class for configuring the Nvidia TensorRT execution provider. It extends StringConfigProviderOptions, providing a string key-value configuration interface. Options include device ID, precision modes, workspace size, engine caching, and other TensorRT-specific parameters.

Description

The class provides:

• Constructors: Default constructor targets device 0; OrtTensorRTProviderOptions(int deviceId) accepts a non-negative device ID.
• Configuration: Inherits add(String key, String value) from StringConfigProviderOptions for settings like "trt_max_workspace_size", "trt_fp16_enable", "trt_int8_enable", "trt_engine_cache_enable", etc.
• Provider identification: getProvider() returns OrtProvider.TENSOR_RT.
• Native integration: Creates native OrtTensorRTProviderOptionsV2 via JNI.

The constructor automatically loads the TensorRT shared library before creating native options.

Code Reference

Source Location

// File: java/src/main/java/ai/onnxruntime/providers/OrtTensorRTProviderOptions.java
// Package: ai.onnxruntime.providers

Signature

public final class OrtTensorRTProviderOptions extends StringConfigProviderOptions {
    public OrtTensorRTProviderOptions() throws OrtException;
    public OrtTensorRTProviderOptions(int deviceId) throws OrtException;
    public OrtProvider getProvider();
}

Import

import ai.onnxruntime.providers.OrtTensorRTProviderOptions;

I/O Contract

Inputs

Name	Type	Description
deviceId	int	CUDA device index (non-negative, default 0)
key	String	Configuration option name (e.g., "trt_fp16_enable")
value	String	Configuration option value

Outputs

Name	Type	Description
OrtTensorRTProviderOptions	OrtTensorRTProviderOptions	Configured TensorRT provider options
getProvider()	OrtProvider	Always OrtProvider.TENSOR_RT

Usage Examples

import ai.onnxruntime.*;
import ai.onnxruntime.providers.OrtTensorRTProviderOptions;

OrtEnvironment env = OrtEnvironment.getEnvironment();

try (OrtTensorRTProviderOptions trtOpts = new OrtTensorRTProviderOptions(0)) {
    trtOpts.add("trt_fp16_enable", "1");
    trtOpts.add("trt_max_workspace_size", String.valueOf(1L << 30)); // 1 GB
    trtOpts.add("trt_engine_cache_enable", "1");
    trtOpts.add("trt_engine_cache_path", "/tmp/trt_cache");

    try (OrtSession.SessionOptions sessionOpts = new OrtSession.SessionOptions()) {
        sessionOpts.addTensorrt(trtOpts);

        try (OrtSession session = env.createSession("/path/to/model.onnx", sessionOpts)) {
            // Session uses TensorRT with fp16 and engine caching
        }
    }
}

Related Pages

• OrtCUDAProviderOptions.java - Similar options for CUDA
• OrtSession.java - SessionOptions.addTensorrt() accepts these options
• OrtProvider.java - TENSOR_RT provider constant