Implementation:Apache Flink HybridSourceEnumeratorState Snapshot

Knowledge Sources	Apache Flink
Domains	Fault_Tolerance, Source_Architecture
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for checkpointing and restoring hybrid source state across source boundaries provided by the Apache Flink connector-base module.

Description

HybridSourceEnumeratorState wraps the current source index and the serialized state of the active underlying enumerator. HybridSourceSplit wraps source-specific splits with their source index and serialized split bytes. The SwitchedSources class maintains a mapping from source indices to their serializers for deserialization during recovery.

On recovery, HybridSource.restoreEnumerator recreates the enumerator at the checkpointed source index with the deserialized underlying state.

Usage

This is an internal mechanism that operates transparently when checkpointing is enabled.

Code Reference

Source Location

• Repository: Apache Flink
• File: flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/hybrid/HybridSourceEnumeratorState.java
• Lines: L22-45
• File: flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/hybrid/HybridSourceSplit.java
• Lines: L31-127

Signature

public class HybridSourceEnumeratorState {
    HybridSourceEnumeratorState(int currentSourceIndex, byte[] wrappedStateBytes, int serializerVersion);
    public int getCurrentSourceIndex();
    public byte[] getWrappedState();
    public int getWrappedStateSerializerVersion();
}

public class HybridSourceSplit implements SourceSplit {
    public HybridSourceSplit(int sourceIndex, byte[] wrappedSplit, int serializerVersion, String splitId);
    public int sourceIndex();
    public byte[] wrappedSplitBytes();
    public String splitId();
    public static List<HybridSourceSplit> wrapSplits(List<? extends SourceSplit> state, int readerIndex, SwitchedSources switchedSources);
    public static List<SourceSplit> unwrapSplits(List<HybridSourceSplit> splits, SwitchedSources switchedSources);
}

Import

import org.apache.flink.connector.base.source.hybrid.HybridSourceEnumeratorState;
import org.apache.flink.connector.base.source.hybrid.HybridSourceSplit;

I/O Contract

Inputs

Name	Type	Required	Description
currentSourceIndex	int	Yes	Active source index in the chain
wrappedStateBytes	byte[]	Yes	Serialized underlying enumerator state
serializerVersion	int	Yes	Version of the state serializer

Outputs

Name	Type	Description
state	HybridSourceEnumeratorState	Checkpoint state for persistence
splits	List<HybridSourceSplit>	Wrapped splits with source index

Usage Examples

Checkpoint and Recovery

// Checkpoint:
// enumerator.snapshotState(checkpointId) produces:
//   HybridSourceEnumeratorState(
//     currentSourceIndex = 1,  // reading from source[1]
//     wrappedStateBytes = serialize(source[1].enumerator.snapshotState()),
//     serializerVersion = 1
//   )
//
// Recovery:
// HybridSource.restoreEnumerator(context, checkpoint) creates:
//   new HybridSourceSplitEnumerator(context, sources, sourceIndex=1, checkpoint)
//   -> Restores source[1]'s enumerator with deserialized state

Related Pages

Implements Principle

• Principle:Apache_Flink_Cross_Source_Checkpointing