Implementation:Apache Flink FileWriter Write

Knowledge Sources	Apache Flink
Domains	Stream_Processing, File_IO
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for writing individual records to bucket-specific part files provided by the Apache Flink connector-files module.

Description

The FileWriter class implements StatefulSinkWriter and CommittingSinkWriter, serving as the runtime writer for the file sink. The write method is the hot-path for data ingestion: it updates the bucket context with temporal metadata, delegates to BucketAssigner.getBucketId to determine the target bucket, retrieves or creates a FileWriterBucket, and writes the element to the buckets in-progress file. It maintains a Map<String, FileWriterBucket<IN>> of active buckets and tracks a records-out counter for metrics.

Usage

This is an internal class invoked automatically by the Flink runtime for each record flowing through the file sink. Users do not call write directly but configure it indirectly through the FileSink builder.

Code Reference

Source Location

• Repository: Apache Flink
• File: flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/writer/FileWriter.java
• Lines: L189-200

Signature

@Internal
public class FileWriter<IN>
        implements StatefulSinkWriter<IN, FileWriterBucketState>,
                CommittingSinkWriter<IN, FileSinkCommittable>,
                SinkWriter<IN>,
                ProcessingTimeService.ProcessingTimeCallback {

    @Override
    public void write(IN element, Context context) throws IOException, InterruptedException {
        bucketerContext.update(
                context.timestamp(),
                context.currentWatermark(),
                processingTimeService.getCurrentProcessingTime());
        final String bucketId = bucketAssigner.getBucketId(element, bucketerContext);
        final FileWriterBucket<IN> bucket = getOrCreateBucketForBucketId(bucketId);
        bucket.write(element, processingTimeService.getCurrentProcessingTime());
        numRecordsOutCounter.inc();
    }
}

Import

import org.apache.flink.connector.file.sink.writer.FileWriter;
// Internal class - not directly imported by users

I/O Contract

Inputs

Name	Type	Required	Description
element	IN	Yes	The record to write
context	SinkWriter.Context	Yes	Provides timestamp() and currentWatermark()

Outputs

Name	Type	Description
side effect	File bytes	Record serialized and appended to the in-progress part file
metrics	Counter increment	numRecordsOutCounter incremented

Usage Examples

Internal Usage (within Flink runtime)

// FileWriter is created internally by FileSink.createWriter()
// Each record in the pipeline triggers:
//   fileWriter.write(element, context);
//
// The write method:
// 1. Gets bucket ID from BucketAssigner
// 2. Routes to the correct FileWriterBucket
// 3. Appends the record to the in-progress file
// 4. Increments the records-out counter

Related Pages

Implements Principle

• Principle:Apache_Flink_Bucket_Data_Writing