Implementation:Apache Hudi OptionsResolver Query Configuration
| Knowledge Sources | |
|---|---|
| Domains | Data_Lake, Stream_Processing |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Concrete tool for determining whether a Flink read query is incremental based on connector configuration, provided by Apache Hudi.
Description
The OptionsResolver.isIncrementalQuery method inspects the Flink Configuration object to determine whether the user has specified an incremental read. It checks for the presence of two optional configuration keys: FlinkOptions.READ_START_COMMIT and FlinkOptions.READ_END_COMMIT. If either is present, the query is classified as incremental.
This method is a static utility that sits in the OptionsResolver class, which centralizes all configuration resolution logic for the Hudi Flink connector. It is called early in the read pipeline -- typically during HoodieTableSource.getScanRuntimeProvider -- to influence which source operator topology and split planning strategy is used.
The method uses Configuration.getOptional() to avoid default-value interference: it does not check whether the values are valid timestamps, only whether they have been explicitly set by the user.
Usage
Use this method when you need to programmatically determine the query type in the Hudi Flink connector. Common call sites include:
HoodieTableSource.getScanRuntimeProvider()to select between streaming, incremental, or snapshot source topologies- Split discovery logic to decide which timeline instants to include
- Test harnesses that verify connector behavior under different read modes
Code Reference
Source Location
- Repository: Apache Hudi
- File:
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java - Lines: 391-393
Signature
public static boolean isIncrementalQuery(Configuration conf) {
return conf.getOptional(FlinkOptions.READ_START_COMMIT).isPresent()
|| conf.getOptional(FlinkOptions.READ_END_COMMIT).isPresent();
}
Import
import org.apache.hudi.configuration.OptionsResolver;
import org.apache.flink.configuration.Configuration;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| conf | org.apache.flink.configuration.Configuration |
Yes | Flink configuration object containing all Hudi connector options, including FlinkOptions.READ_START_COMMIT and FlinkOptions.READ_END_COMMIT.
|
Outputs
| Name | Type | Description |
|---|---|---|
| return value | boolean |
true if either READ_START_COMMIT or READ_END_COMMIT is explicitly set in the configuration; false otherwise.
|
Usage Examples
import org.apache.hudi.configuration.OptionsResolver;
import org.apache.hudi.configuration.FlinkOptions;
import org.apache.flink.configuration.Configuration;
// Example: Check if the query is incremental
Configuration conf = new Configuration();
conf.setString(FlinkOptions.READ_START_COMMIT, "20240101120000000");
conf.setString(FlinkOptions.READ_END_COMMIT, "20240102120000000");
boolean isIncremental = OptionsResolver.isIncrementalQuery(conf);
// isIncremental == true
// Example: Snapshot query (no commit range specified)
Configuration snapshotConf = new Configuration();
boolean isSnapshotIncremental = OptionsResolver.isIncrementalQuery(snapshotConf);
// isSnapshotIncremental == false