Implementation:Apache Flink CheckpointedPosition
| Knowledge Sources | |
|---|---|
| Domains | Connectors, Source_Framework |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
An immutable value class representing a reader's checkpoint position, consisting of an optional offset and a number of records to skip after that offset.
Description
CheckpointedPosition is a public-evolving, serializable class that encapsulates the position information needed to resume reading from a checkpoint. It combines two components:
- offset: A seek position that the reader will use to position itself when restoring from a checkpoint. This can be a byte offset, block position, or any addressable position depending on the format. The special value NO_OFFSET (-1) indicates that no offset information is available.
- recordsAfterOffset: The number of records to skip after seeking to the offset. This allows precise positioning even when the format cannot address individual records directly.
This two-part position scheme supports a wide variety of reader implementations:
- Simple readers: Store no offset (NO_OFFSET) and only track the total number of records returned.
- Precise readers: Store an exact byte offset for each record with zero records to skip.
- Block-based readers: Store the position of the last addressable marker (e.g., sync marker or block start) along with the number of records consumed after that marker.
The class implements equals(), hashCode(), and toString() for proper use in collections and debugging.
Usage
Use CheckpointedPosition to represent and store the position of a file source reader in checkpoint state. It is used by RecordAndPosition and MutableRecordAndPosition to communicate position information through the BulkFormat.RecordIterator interface. Format implementors use this class when defining how their reader positions should be serialized into checkpoints.
Code Reference
Source Location
- Repository: Apache_Flink
- File:
flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/src/util/CheckpointedPosition.java - Lines: 1-102
Signature
@PublicEvolving
public final class CheckpointedPosition implements Serializable
Import
import org.apache.flink.connector.file.src.util.CheckpointedPosition;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| offset | long | Yes | The seek offset for checkpoint restoration; must be >= 0 or NO_OFFSET (-1) |
| recordsAfterOffset | long | Yes | The number of records to skip after seeking to the offset; must be >= 0 |
Outputs
| Name | Type | Description |
|---|---|---|
| getOffset() | long | The stored offset value, or NO_OFFSET (-1) if not applicable |
| getRecordsAfterOffset() | long | The number of records to skip after the offset |
Usage Examples
// Create a position with a specific byte offset and records to skip
CheckpointedPosition position = new CheckpointedPosition(4096L, 3L);
long offset = position.getOffset(); // 4096
long toSkip = position.getRecordsAfterOffset(); // 3
// Create a position without offset information (record-count only)
CheckpointedPosition countOnly =
new CheckpointedPosition(CheckpointedPosition.NO_OFFSET, 150L);
// Use in equality checks
CheckpointedPosition pos1 = new CheckpointedPosition(100L, 5L);
CheckpointedPosition pos2 = new CheckpointedPosition(100L, 5L);
assert pos1.equals(pos2); // true