Implementation:Apache Flink FileCommitter Commit

Knowledge Sources	Apache Flink
Domains	Stream_Processing, Fault_Tolerance
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for finalizing pending files by committing them from a recoverable state to their permanent location provided by the Apache Flink connector-files module.

Description

The FileCommitter class implements Committer<FileSinkCommittable> and handles the commit phase of the two-phase commit protocol. For each CommitRequest, it processes three types of operations: pending file commit (recover and finalize), in-progress file cleanup (discard incomplete files), and compacted file cleanup (delete pre-compaction source files). All operations are idempotent.

Usage

This is an internal class invoked automatically by the Flink sink framework after a checkpoint succeeds. Users do not interact with it directly.

Code Reference

Source Location

• Repository: Apache Flink
• File: flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/committer/FileCommitter.java
• Lines: L45-91

Signature

@Internal
public class FileCommitter implements Committer<FileSinkCommittable> {

    public FileCommitter(BucketWriter<?, ?> bucketWriter);

    @Override
    public void commit(Collection<CommitRequest<FileSinkCommittable>> requests)
            throws IOException, InterruptedException {
        for (CommitRequest<FileSinkCommittable> request : requests) {
            FileSinkCommittable committable = request.getCommittable();
            if (committable.hasPendingFile()) {
                bucketWriter.recoverPendingFile(committable.getPendingFile())
                    .commitAfterRecovery();
            }
            if (committable.hasInProgressFileToCleanup()) {
                bucketWriter.cleanupInProgressFileRecoverable(
                    committable.getInProgressFileToCleanup());
            }
            if (committable.hasCompactedFileToCleanup()) {
                Path fileToCleanup = committable.getCompactedFileToCleanup();
                fileToCleanup.getFileSystem().delete(fileToCleanup, false);
            }
        }
    }

    @Override
    public void close() throws Exception {}
}

Import

import org.apache.flink.connector.file.sink.committer.FileCommitter;
// Internal class

I/O Contract

Inputs

Name	Type	Required	Description
requests	Collection<CommitRequest<FileSinkCommittable>>	Yes	Committable descriptors from prepareCommit
bucketWriter	BucketWriter<?, ?>	Yes	Writer capable of recovering pending files

Outputs

Name	Type	Description
side effect	File state changes	Pending files finalized, in-progress files cleaned up, compacted files deleted

Usage Examples

Commit Lifecycle

// After checkpoint N succeeds, Flink calls:
// fileCommitter.commit(committableRequests);
//
// For each request:
// 1. If hasPendingFile() -> recover and commit (rename temp -> final)
// 2. If hasInProgressFileToCleanup() -> delete stale temp file
// 3. If hasCompactedFileToCleanup() -> delete pre-compaction source
//
// All operations are idempotent for exactly-once semantics

Related Pages

Implements Principle

• Principle:Apache_Flink_File_Commit_Finalization