Implementation:Microsoft Onnxruntime OrtModelCompilationOptions

Knowledge Sources	Description
Source File	java/src/main/java/ai/onnxruntime/OrtModelCompilationOptions.java
Repository	Microsoft/onnxruntime

Domains

• Machine Learning Runtime
• Model Compilation
• Execution Provider Context

Overview

OrtModelCompilationOptions is a final class providing configuration for compiling ONNX models with execution provider context. It allows setting input/output model paths, external initializer storage, EP context embedding, and compilation flags. The compilation generates an optimized model with EPContext nodes that embed or reference execution provider caches.

Description

The class provides:

• Factory method: createFromSessionOptions(OrtEnvironment, SessionOptions) creates an options instance from existing session options (which specify the execution providers).
• Input model: setInputModelPath(String) or setInputModelFromBuffer(ByteBuffer) -- one must be set.
• Output model: setOutputModelPath(String) -- optional, defaults to appending "_ctx" to the input filename.
• External initializers: setOutputExternalInitializersPath(String, long) stores large initializers in a separate file.
• EP context embedding: setEpContextEmbedMode(boolean) controls whether EPContext binary data is embedded in the model or stored externally.
• Compilation flags: setCompilationFlags(EnumSet<OrtCompileApiFlags>) with flags ERROR_IF_NO_NODES_COMPILED and ERROR_IF_OUTPUT_FILE_EXISTS.
• Compilation: compileModel() performs the actual compilation.

The OrtCompileApiFlags inner enum defines: NONE (0), ERROR_IF_NO_NODES_COMPILED (1), ERROR_IF_OUTPUT_FILE_EXISTS (2).

Code Reference

Source Location

// File: java/src/main/java/ai/onnxruntime/OrtModelCompilationOptions.java
// Package: ai.onnxruntime

Signature

public final class OrtModelCompilationOptions implements AutoCloseable {
    public static OrtModelCompilationOptions createFromSessionOptions(
        OrtEnvironment env, OrtSession.SessionOptions sessionOptions) throws OrtException;

    public void setInputModelPath(String inputModelPath) throws OrtException;
    public void setInputModelFromBuffer(ByteBuffer inputModelBuffer) throws OrtException;
    public void setOutputModelPath(String outputModelPath) throws OrtException;
    public void setOutputExternalInitializersPath(
        String outputExternalInitializersPath, long sizeThreshold) throws OrtException;
    public void setEpContextEmbedMode(boolean embedEpContext) throws OrtException;
    public void setCompilationFlags(EnumSet<OrtCompileApiFlags> flags) throws OrtException;
    public void compileModel() throws OrtException;
    public void close();

    public enum OrtCompileApiFlags implements OrtFlags {
        NONE(0), ERROR_IF_NO_NODES_COMPILED(1), ERROR_IF_OUTPUT_FILE_EXISTS(2);
    }
}

Import

import ai.onnxruntime.OrtModelCompilationOptions;
import ai.onnxruntime.OrtModelCompilationOptions.OrtCompileApiFlags;

I/O Contract

Inputs

Name	Type	Description
env	OrtEnvironment	The ONNX Runtime environment
sessionOptions	SessionOptions	Session options specifying execution providers
inputModelPath	String	Path to the input ONNX model
outputModelPath	String	Path for the compiled output model
flags	EnumSet<OrtCompileApiFlags>	Compilation behavior flags

Outputs

Name	Type	Description
compileModel()	void	Writes the compiled model to the output path

Usage Examples

import ai.onnxruntime.*;
import java.util.EnumSet;

OrtEnvironment env = OrtEnvironment.getEnvironment();

try (OrtSession.SessionOptions opts = new OrtSession.SessionOptions()) {
    // Configure execution providers for compilation
    opts.addCUDA(0);

    try (OrtModelCompilationOptions compileOpts =
            OrtModelCompilationOptions.createFromSessionOptions(env, opts)) {
        compileOpts.setInputModelPath("/path/to/model.onnx");
        compileOpts.setOutputModelPath("/path/to/model_compiled.onnx");
        compileOpts.setEpContextEmbedMode(true);
        compileOpts.setCompilationFlags(
            EnumSet.of(OrtModelCompilationOptions.OrtCompileApiFlags.ERROR_IF_NO_NODES_COMPILED));

        compileOpts.compileModel();
        System.out.println("Model compiled successfully.");
    }
}

Related Pages

• OrtSession.java - SessionOptions used as compilation input
• OrtEnvironment.java - Environment required for compilation