Implementation:Apache Paimon TableCommit Commit
| Knowledge Sources | |
|---|---|
| Domains | Data_Lake, Table_Format |
| Last Updated | 2026-02-07 00:00 GMT |
Overview
Concrete tool for atomically committing batch writes to Paimon tables via snapshot creation.
Description
BatchTableCommit.commit() takes a list of CommitMessage objects from prepare_commit() and creates an atomic Snapshot. It delegates to FileStoreCommit which handles the actual snapshot creation, manifest updates, and cleanup. The implementation supports both regular commits and partition-level overwrites via truncate_partitions(). One-time commit semantics prevent accidental double commits by raising an error if commit() is called more than once on the same instance.
Usage
Use this implementation after calling prepare_commit() on a BatchTableWrite instance. Create a commit object via write_builder.new_commit(), then call commit() with the commit messages to finalize the write. After committing, the data becomes visible to all subsequent readers.
Code Reference
Source Location
- Repository: Apache Paimon
- File: paimon-python/pypaimon/write/table_commit.py
- Lines: L26-91
Signature
class TableCommit:
def __init__(self, table, commit_user: str, static_partition: Optional[dict]):
class BatchTableCommit(TableCommit):
def commit(self, commit_messages: List[CommitMessage]):
def truncate_partitions(self, partitions: List[Dict[str, str]]) -> None:
Import
from pypaimon.write.table_commit import BatchTableCommit
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| commit_messages | List[CommitMessage] | Yes | List of commit messages obtained from BatchTableWrite.prepare_commit()
|
| partitions | List[Dict[str, str]] | Yes (for truncate_partitions) | List of partition specifications to truncate |
Outputs
| Name | Type | Description |
|---|---|---|
| commit return | None | Creates an atomic Snapshot in storage; data becomes visible to readers (side effect) |
| truncate_partitions return | None | Removes data from the specified partitions (side effect) |
Usage Examples
Basic Usage
# After writing data
commit_messages = writer.prepare_commit()
# Create commit and finalize
commit_builder = write_builder.new_commit()
commit_builder.commit(commit_messages)