Principle:Apache Flink Compaction Request Triggering
| Knowledge Sources | |
|---|---|
| Domains | Stream_Processing, File_IO |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A request encapsulation mechanism that packages per-bucket file lists into serializable compaction requests for downstream processing.
Description
Compaction Request Triggering packages the coordinators accumulated files into CompactorRequest objects. Each request contains:
- bucketId: The bucket being compacted
- committableToCompact: Files to merge together
- committableToPassthrough: Files to forward unchanged (cleanup operations)
The request is serialized by CompactorRequestSerializer (using a magic number for validation) and emitted as a StreamRecord to the compactor operator.
Usage
This principle is internal to the compaction pipeline, serving as the data contract between the coordinator and the compactor operator.
Theoretical Basis
// Abstract request formation
function createRequest(bucketId, filesToCompact, filesToPassthrough):
request = new CompactorRequest(bucketId)
for each file in filesToCompact:
request.addToCompact(file)
for each file in filesToPassthrough:
request.addToPassthrough(file)
return request
Related Pages
Implemented By
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment