Principle:Onnx Onnx External Data Verification
| Knowledge Sources | |
|---|---|
| Domains | External_Data, Validation |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
A path-based validation mechanism that checks the integrity of ONNX models with external data by operating directly on file paths to avoid the 2GB protobuf memory limit.
Description
When an ONNX model with external data exceeds the 2GB protobuf size limit, it cannot be loaded entirely into memory for validation. Path-based verification solves this by having the C++ checker and shape inference engines read the model file and external data directly from disk, bypassing the protobuf size constraint. The check_model function's path-based branch calls the C++ check_model_path function, and infer_shapes_path performs shape inference directly on disk.
This verification confirms that the model structure is valid, external data references resolve correctly, and the model with full tensor data passes all ONNX specification checks.
Usage
Use this principle as the final step in the external data handling workflow, after saving a model with external data. Path-based verification is required for models whose total tensor data exceeds 2GB. For smaller models, in-memory check_model with a ModelProto is sufficient.
Theoretical Basis
Path-based verification avoids the 2GB memory limit:
Failed to parse (syntax error): {\displaystyle \text{verify\_path}(p) = \text{check\_model\_path}(p) \land \text{infer\_shapes\_path}(p) }
The C++ engine handles the file I/O directly, loading only the portions of tensor data needed for each validation check.