Implementation:Apache Flink RecordAndPosition
| Knowledge Sources | |
|---|---|
| Domains | Connectors, File_Connector |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
An immutable container that pairs a record with the reader position to be stored in a checkpoint, representing the point in the reader after the record has been processed.
Description
RecordAndPosition is a generic class that bundles a record of type E together with checkpoint position information consisting of an offset and a recordSkipCount. The position semantics are critical: the stored position defines the point in the reader after the record, so that upon recovery from a checkpoint, the reader resumes from exactly after this record. For example, the very first record in a file split would have an offset of zero and a recordSkipCount of one.
This design supports formats where skipping records is not feasible (such as formats with pushed-down filters), because the position always points to where the reader should continue rather than requiring post-recovery record skipping.
The class is immutable for safety. For mutable use cases where object reuse is desired for efficiency, the subclass MutableRecordAndPosition should be used instead. The fields are package-private and non-final to allow mutation from the mutable subclass.
The constant NO_OFFSET (delegated from CheckpointedPosition.NO_OFFSET) indicates that position information does not include an offset and is defined only by a record skip count.
Usage
Use RecordAndPosition in BulkFormat.RecordIterator implementations to return records together with their checkpoint-safe position information. This is the standard return type from record iterators in the Flink file source framework.
Code Reference
Source Location
- Repository: Apache_Flink
- File: flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/src/util/RecordAndPosition.java
- Lines: 1-87
Signature
@PublicEvolving
public class RecordAndPosition<E> {
public static final long NO_OFFSET = CheckpointedPosition.NO_OFFSET;
public RecordAndPosition(E record, long offset, long recordSkipCount);
RecordAndPosition();
public E getRecord();
public long getOffset();
public long getRecordSkipCount();
@Override
public String toString();
}
Import
import org.apache.flink.connector.file.src.util.RecordAndPosition;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| record | E | Yes | The record element to associate with this position. |
| offset | long | Yes | The offset in the file split after this record. Use NO_OFFSET when the position is defined only by skip count. |
| recordSkipCount | long | Yes | The number of records to skip from the offset to reach the position after this record. |
Outputs
| Name | Type | Description |
|---|---|---|
| getRecord() | E | The stored record element. |
| getOffset() | long | The offset position after this record. |
| getRecordSkipCount() | long | The record skip count for the position after this record. |
Usage Examples
// Create a RecordAndPosition for the first record in a split
// offset=0, skipCount=1 means: after processing, resume at offset 0 skipping 1 record
RecordAndPosition<RowData> recordAndPos = new RecordAndPosition<>(rowData, 0L, 1L);
// Access the record and position
RowData record = recordAndPos.getRecord();
long offset = recordAndPos.getOffset();
long skipCount = recordAndPos.getRecordSkipCount();
// Use NO_OFFSET when position is purely skip-count based
RecordAndPosition<RowData> skipOnly = new RecordAndPosition<>(
rowData, RecordAndPosition.NO_OFFSET, 5L);