Implementation:Datahub project Datahub ProtobufDescriptorUtils

Knowledge Sources	Datahub_project_Datahub
Domains	Protobuf_Integration, Schema_Processing
Last Updated	2026-02-10 00:00 GMT

Overview

Description

ProtobufUtils is a utility class in the DataHub protobuf integration module that provides low-level operations for working with Protocol Buffer descriptors. It has three primary responsibilities:

1. Comment extraction -- Collapses leading, trailing, and detached comments from SourceCodeInfo.Location into a single trimmed string.
2. Option extraction -- Retrieves field-level and message-level options (including custom extensions) from descriptor proto objects. This uses Java reflection to work around a known protobuf library bug where calling certain getter methods mutates internal state, causing IllegalArgumentException on subsequent comparisons.
3. Extension registry building -- Constructs an ExtensionRegistry from a FileDescriptorSet by recursively resolving file descriptor dependencies and registering all extensions (including nested message-type extensions via DynamicMessage).

Usage

Used internally by other protobuf visitor and model classes to extract metadata (comments, options, extensions) from protobuf descriptors during the schema-to-DataHub conversion process.

Code Reference

Source Location

metadata-integration/java/datahub-protobuf/src/main/java/datahub/protobuf/ProtobufUtils.java

Signature

public class ProtobufUtils {

    public static String collapseLocationComments(
        DescriptorProtos.SourceCodeInfo.Location location)

    public static List<Pair<Descriptors.FieldDescriptor, Object>> getFieldOptions(
        DescriptorProtos.FieldDescriptorProto fieldProto)

    public static List<Pair<Descriptors.FieldDescriptor, Object>> getMessageOptions(
        DescriptorProtos.DescriptorProto messageProto)

    public static ExtensionRegistry buildRegistry(
        DescriptorProtos.FileDescriptorSet fileSet)
}

Import

import datahub.protobuf.ProtobufUtils;

I/O Contract

Inputs

Method	Parameter	Type	Description
collapseLocationComments	location	SourceCodeInfo.Location	A source code location from a proto file descriptor
getFieldOptions	fieldProto	FieldDescriptorProto	A field descriptor proto whose options are to be extracted
getMessageOptions	messageProto	DescriptorProto	A message descriptor proto whose options are to be extracted
buildRegistry	fileSet	FileDescriptorSet	A set of file descriptors from a compiled protobuf

Outputs

Method	Return Type	Description
collapseLocationComments	String	Concatenated and trimmed comments from the location
getFieldOptions	List<Pair<FieldDescriptor, Object>>	All options (standard and extension) for the field
getMessageOptions	List<Pair<FieldDescriptor, Object>>	All options (standard and extension) for the message
buildRegistry	ExtensionRegistry	A populated extension registry with all extensions from the file set

Usage Examples

// Collapse protobuf source comments
String comment = ProtobufUtils.collapseLocationComments(location);

// Get all options for a field descriptor proto
List<Pair<Descriptors.FieldDescriptor, Object>> fieldOpts =
    ProtobufUtils.getFieldOptions(fieldDescriptorProto);

// Build an extension registry from a file descriptor set
ExtensionRegistry registry = ProtobufUtils.buildRegistry(fileDescriptorSet);

Related Pages

• Datahub_project_Datahub_ProtobufMessage -- Uses ProtobufUtils for comment extraction
• Datahub_project_Datahub_ProtobufExtensionUtil -- Builds on ProtobufUtils option extraction
• Datahub_project_Datahub_ProtobufExtensionFieldVisitor -- Consumes field and message options from ProtobufUtils