Implementation:Apache Paimon BinaryRow
| Knowledge Sources | |
|---|---|
| Domains | Row Representation, Binary Serialization |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
BinaryRow is a compact binary format for storing row data that provides efficient memory usage and fast field access.
Description
The BinaryRow class implements the InternalRow interface with a binary serialization format optimized for performance and memory efficiency. It stores row data in a compact byte array with a fixed-size header containing field count (arity) and a bit-set for null value tracking.
The binary format includes a 4-byte arity prefix followed by a 1-byte row kind indicator and a variable-length bit-set for tracking null values. Field values are stored after the bit-set with offsets calculated based on field positions. The implementation uses GenericRowDeserializer for parsing individual field values based on their data types.
BinaryRow provides constant-time field access by index with null checking and proper type deserialization. It automatically handles the arity prefix and exposes the actual data portion separately for operations that need raw binary data.
Usage
Use BinaryRow when working with serialized row data in manifest files, statistics, or when you need memory-efficient row storage without the overhead of Python objects for each field.
Code Reference
Source Location
- Repository: Apache_Paimon
- File: paimon-python/pypaimon/table/row/binary_row.py
Signature
class BinaryRow(InternalRow):
"""BinaryRow is a compact binary format for storing a row of data."""
def __init__(self, data: bytes, fields: List[DataField]):
"""Initialize BinaryRow with raw binary data and field definitions."""
def get_field(self, index: int) -> Any:
"""Get field value by index."""
def get_row_kind(self) -> RowKind:
"""Get row kind."""
def __len__(self):
"""Return number of fields."""
Import
from pypaimon.table.row.binary_row import BinaryRow
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| data | bytes | Yes | Raw binary row data |
| fields | List[DataField] | Yes | Field definitions for deserialization |
| index | int | Yes (for get_field) | Field index to retrieve |
Outputs
| Name | Type | Description |
|---|---|---|
| field_value | Any | Deserialized field value (or None if null) |
| row_kind | RowKind | Row change kind (INSERT, UPDATE, DELETE) |
| arity | int | Number of fields in the row |
Usage Examples
from pypaimon.table.row.binary_row import BinaryRow
from pypaimon.schema.data_types import DataField, AtomicType
# Define fields
fields = [
DataField(0, "id", AtomicType("BIGINT")),
DataField(1, "name", AtomicType("STRING")),
DataField(2, "age", AtomicType("INT"))
]
# Create BinaryRow from binary data
binary_data = b'\x00\x00\x00\x03...' # Binary row data
row = BinaryRow(binary_data, fields)
# Access fields
id_value = row.get_field(0)
name_value = row.get_field(1)
age_value = row.get_field(2)
print(f"ID: {id_value}, Name: {name_value}, Age: {age_value}")
# Check row kind
row_kind = row.get_row_kind()
print(f"Row kind: {row_kind.to_string()}")
# Get number of fields
num_fields = len(row)
# Handle null values
if row.get_field(1) is None:
print("Name is null")
# Use in statistics (common use case)
from pypaimon.manifest.schema.simple_stats import SimpleStats
# min_values and max_values in SimpleStats are often BinaryRow
stats = SimpleStats(
min_values=BinaryRow(min_data, key_fields),
max_values=BinaryRow(max_data, key_fields),
null_counts=[0, 1, 0]
)
min_key = stats.min_values.get_field(0)
max_key = stats.max_values.get_field(0)