Implementation:Apache Paimon Partition
| Knowledge Sources | |
|---|---|
| Domains | Data Management, Partition Management |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
The Partition class represents a table partition in Apache Paimon, including comprehensive statistics and metadata about the partition's state and lifecycle.
Description
Partition extends PartitionStatistics to provide a complete representation of a partition within the Paimon table format. It includes not only statistical information such as record count, file size, and file count, but also tracks the partition's completion status (done flag) and maintains audit metadata about when and by whom the partition was created or modified.
The class is designed to be serializable and JSON-compatible, making it suitable for use in REST APIs and distributed systems. It supports tracking of partition specifications, statistics, and optional metadata including creation timestamps, update timestamps, and custom options. The done flag indicates whether the partition has finished loading or processing.
This implementation is part of the public API and uses Jackson annotations for JSON serialization and deserialization, allowing seamless integration with REST-based catalog services.
Usage
Use the Partition class when working with Apache Paimon partitioned tables to represent partition metadata and statistics. It's particularly useful for REST catalog operations, partition management tasks, and when tracking the lifecycle and statistics of individual partitions within a table.
Code Reference
Source Location
Signature
@JsonIgnoreProperties(ignoreUnknown = true)
@Public
public class Partition extends PartitionStatistics {
private static final long serialVersionUID = 3L;
@JsonProperty(FIELD_DONE)
private final boolean done;
@JsonProperty(FIELD_CREATED_AT)
@JsonInclude(JsonInclude.Include.NON_NULL)
@Nullable
private final Long createdAt;
@JsonProperty(FIELD_CREATED_BY)
@JsonInclude(JsonInclude.Include.NON_NULL)
@Nullable
private final String createdBy;
@JsonProperty(FIELD_UPDATED_AT)
@JsonInclude(JsonInclude.Include.NON_NULL)
@Nullable
private final Long updatedAt;
@JsonProperty(FIELD_UPDATED_BY)
@JsonInclude(JsonInclude.Include.NON_NULL)
@Nullable
private final String updatedBy;
@JsonProperty(FIELD_OPTIONS)
@JsonInclude(JsonInclude.Include.NON_NULL)
@Nullable
private final Map<String, String> options;
}
Import
import org.apache.paimon.partition.Partition;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| spec | Map<String, String> | Yes | Partition specification mapping column names to values |
| recordCount | long | Yes | Number of records in the partition |
| fileSizeInBytes | long | Yes | Total file size in bytes |
| fileCount | long | Yes | Number of files in the partition |
| lastFileCreationTime | long | Yes | Timestamp of last file creation |
| totalBuckets | int | Yes | Total number of buckets |
| done | boolean | Yes | Flag indicating if partition is complete |
| createdAt | Long | No | Timestamp when partition was created (milliseconds) |
| createdBy | String | No | User or system that created the partition |
| updatedAt | Long | No | Timestamp when partition was last updated (milliseconds) |
| updatedBy | String | No | User or system that last updated the partition |
| options | Map<String, String> | No | Custom options for the partition |
Outputs
| Name | Type | Description |
|---|---|---|
| done() | boolean | Returns whether the partition is marked as done/complete |
| createdAt() | Long | Returns the creation timestamp (may be null) |
| createdBy() | String | Returns the creator identifier (may be null) |
| updatedAt() | Long | Returns the last update timestamp (may be null) |
| updatedBy() | String | Returns the last updater identifier (may be null) |
| options() | Map<String, String> | Returns custom partition options (may be null) |
| spec() | Map<String, String> | Returns partition specification (inherited) |
| recordCount() | long | Returns number of records (inherited) |
| fileSizeInBytes() | long | Returns total file size (inherited) |
| fileCount() | long | Returns number of files (inherited) |
| lastFileCreationTime() | long | Returns last file creation time (inherited) |
| totalBuckets() | int | Returns total bucket count (inherited) |
Usage Examples
// Create a partition with basic statistics
Map<String, String> spec = new HashMap<>();
spec.put("year", "2024");
spec.put("month", "01");
Partition partition = new Partition(
spec,
100000L, // recordCount
1024L * 1024 * 500, // fileSizeInBytes (500 MB)
10L, // fileCount
System.currentTimeMillis(), // lastFileCreationTime
16, // totalBuckets
true // done
);
// Create a partition with audit metadata
Partition partitionWithMetadata = new Partition(
spec,
100000L,
1024L * 1024 * 500,
10L,
System.currentTimeMillis(),
16,
true,
System.currentTimeMillis(), // createdAt
"user@example.com", // createdBy
System.currentTimeMillis(), // updatedAt
"system", // updatedBy
Collections.singletonMap("custom.property", "value") // options
);
// Check if partition is complete
if (partition.done()) {
System.out.println("Partition processing is complete");
}
// Access partition statistics
System.out.println("Records: " + partition.recordCount());
System.out.println("Total size: " + partition.fileSizeInBytes() + " bytes");
System.out.println("Files: " + partition.fileCount());