Principle:Apache Flink Async Sink Configuration
| Knowledge Sources | |
|---|---|
| Domains | Stream_Processing, Configuration |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A builder-based configuration pattern that specifies batching, flow control, and timeout parameters for asynchronous sink connectors.
Description
Async Sink Configuration addresses the challenge of tuning asynchronous I/O for external systems. The configuration parameters control three aspects:
- Batching: Maximum batch size (records and bytes) and maximum buffer time before flushing
- Flow control: Maximum in-flight requests and maximum buffered entries to prevent memory exhaustion
- Reliability: Request timeout and fail-on-timeout behavior
These parameters must be balanced: too-small batches waste network overhead, too-large buffers risk OOM, and too many in-flight requests can overwhelm the destination.
Usage
Use this principle when implementing a custom async sink. Start with conservative defaults and tune based on the destination systems throughput characteristics and Flinks available memory.
Theoretical Basis
// Configuration parameter space
maxBatchSize: int // Max records per batch (destination API limit)
maxBatchSizeInBytes: long // Max bytes per batch
maxInFlightRequests: int // Max concurrent async submissions
maxBufferedRequests: int // Max entries in buffer before blocking
maxTimeInBufferMS: long // Max time before forced flush
maxRecordSizeInBytes: long // Max individual record size
requestTimeoutMS: long // Timeout per async request
failOnTimeout: boolean // Whether timeout is fatal
Related Pages
Implemented By
Uses Heuristic
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment