Implementation:Onnx Onnx Proto Utils
| Knowledge Sources | |
|---|---|
| Domains | Protobuf Utilities, Model Loading |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
Concrete tool for working with Protocol Buffer messages in ONNX, providing debug string conversion, safe large-model parsing, and attribute value extraction, provided by the ONNX library.
Description
This header provides essential utility functions for Protocol Buffer message handling within the ONNX library. It supports both full protobuf and protobuf-lite builds through conditional compilation.
ProtoDebugString returns a short debug string representation of a protobuf message. When compiled with ONNX_USE_LITE_PROTO, it operates on google::protobuf::MessageLite (which has limited reflection capabilities). Otherwise, it operates on the full google::protobuf::Message interface. Both variants call ShortDebugString() on the message.
ParseProtoFromBytes is a template function for safely parsing protobuf messages from raw byte buffers. It creates an ArrayInputStream from the provided buffer and wraps it in a CodedInputStream with a total bytes limit of approximately 2 GB (2048 << 20 - 1). This is critical for handling large ONNX models that exceed protobuf's default size limits. The function includes version-specific handling for the protobuf API change at version 3.11, where SetTotalBytesLimit changed from two parameters to one.
RetrieveValues is a template function with explicit specializations for extracting typed vectors from AttributeProto objects. It supports three types:
- int64_t - extracts from the ints repeated field
- std::string - extracts from the strings repeated field
- float - extracts from the floats repeated field
Usage
Use ProtoDebugString for logging and debugging protobuf messages. Use ParseProtoFromBytes when loading ONNX models from byte arrays, especially large models that may exceed default protobuf size limits. Use RetrieveValues when you need to extract repeated attribute values from operator attributes during model processing.
Code Reference
Source Location
- Repository: Onnx_Onnx
- File: onnx/proto_utils.h
Signature
// Debug string (full protobuf build)
inline std::string ProtoDebugString(const ::google::protobuf::Message& proto);
// Debug string (lite protobuf build)
inline std::string ProtoDebugString(const ::google::protobuf::MessageLite& proto);
// Safe parsing with 2GB limit
template <typename Proto>
bool ParseProtoFromBytes(Proto* proto, const char* buffer, size_t length);
// Attribute value extraction
template <typename T>
inline std::vector<T> RetrieveValues(const AttributeProto& attr);
// Specializations:
template <> inline std::vector<int64_t> RetrieveValues(const AttributeProto& attr);
template <> inline std::vector<std::string> RetrieveValues(const AttributeProto& attr);
template <> inline std::vector<float> RetrieveValues(const AttributeProto& attr);
Import
#include "onnx/proto_utils.h"
I/O Contract
Inputs (ParseProtoFromBytes)
| Name | Type | Required | Description |
|---|---|---|---|
| proto | Proto* | Yes | Pointer to the protobuf message object to populate |
| buffer | const char* | Yes | Raw byte buffer containing the serialized protobuf data |
| length | size_t | Yes | Length of the byte buffer in bytes |
Inputs (RetrieveValues)
| Name | Type | Required | Description |
|---|---|---|---|
| attr | const AttributeProto& | Yes | The ONNX attribute from which to extract repeated values |
Outputs
| Name | Type | Description |
|---|---|---|
| debug_string | std::string | Human-readable short debug string of the protobuf message (from ProtoDebugString) |
| success | bool | Whether parsing succeeded (from ParseProtoFromBytes) |
| values | std::vector<T> | Extracted attribute values as a typed vector (from RetrieveValues) |
Usage Examples
#include "onnx/proto_utils.h"
// Parse an ONNX model from a byte buffer
ModelProto model;
const char* data = ...;
size_t data_size = ...;
bool success = ParseProtoFromBytes(&model, data, data_size);
if (!success) {
// Handle parse error
}
// Get debug string for logging
std::string debug = ProtoDebugString(model);
// Extract repeated attribute values
for (const auto& attr : node.attribute()) {
if (attr.name() == "axes") {
std::vector<int64_t> axes = RetrieveValues<int64_t>(attr);
}
if (attr.name() == "scales") {
std::vector<float> scales = RetrieveValues<float>(attr);
}
}