Implementation:Microsoft Onnxruntime OrtCUDAProviderOptions

Knowledge Sources	Description
Source File	java/src/main/java/ai/onnxruntime/providers/OrtCUDAProviderOptions.java
Repository	Microsoft/onnxruntime

Domains

• Machine Learning Runtime
• GPU Acceleration
• CUDA Configuration

Overview

OrtCUDAProviderOptions is a final class for configuring the CUDA execution provider. It extends StringConfigProviderOptions, providing a string key-value configuration interface. Options include device ID, memory limits, arena settings, and other CUDA-specific parameters documented on the ORT website.

Description

The class provides:

• Constructors: Default constructor targets device 0; OrtCUDAProviderOptions(int deviceId) accepts a non-negative device ID.
• Configuration: Inherits add(String key, String value) from StringConfigProviderOptions for setting options like "gpu_mem_limit", "arena_extend_strategy", "cudnn_conv_algo_search", etc.
• Provider identification: getProvider() returns OrtProvider.CUDA.
• Native integration: Creates native OrtCUDAProviderOptionsV2 via JNI, applies options, and releases resources on close.

The constructor automatically loads the CUDA shared library before creating native options.

Code Reference

Source Location

// File: java/src/main/java/ai/onnxruntime/providers/OrtCUDAProviderOptions.java
// Package: ai.onnxruntime.providers

Signature

public final class OrtCUDAProviderOptions extends StringConfigProviderOptions {
    public OrtCUDAProviderOptions() throws OrtException;
    public OrtCUDAProviderOptions(int deviceId) throws OrtException;
    public OrtProvider getProvider();
}

Import

import ai.onnxruntime.providers.OrtCUDAProviderOptions;

I/O Contract

Inputs

Name	Type	Description
deviceId	int	CUDA device index (non-negative, default 0)
key	String	Configuration option name (e.g., "gpu_mem_limit")
value	String	Configuration option value

Outputs

Name	Type	Description
OrtCUDAProviderOptions	OrtCUDAProviderOptions	Configured CUDA provider options
getProvider()	OrtProvider	Always OrtProvider.CUDA

Usage Examples

import ai.onnxruntime.*;
import ai.onnxruntime.providers.OrtCUDAProviderOptions;

OrtEnvironment env = OrtEnvironment.getEnvironment();

try (OrtCUDAProviderOptions cudaOpts = new OrtCUDAProviderOptions(0)) {
    cudaOpts.add("gpu_mem_limit", String.valueOf(2L * 1024 * 1024 * 1024));
    cudaOpts.add("arena_extend_strategy", "kSameAsRequested");
    cudaOpts.add("cudnn_conv_algo_search", "DEFAULT");

    try (OrtSession.SessionOptions sessionOpts = new OrtSession.SessionOptions()) {
        sessionOpts.addCUDA(cudaOpts);

        try (OrtSession session = env.createSession("/path/to/model.onnx", sessionOpts)) {
            // Session uses CUDA with custom options
        }
    }
}

Related Pages

• OrtTensorRTProviderOptions.java - Similar options for TensorRT
• OrtSession.java - SessionOptions.addCUDA() accepts these options
• OrtProvider.java - CUDA provider constant