Implementation:Apache Spark WriteAheadLogRecordHandle
| Knowledge Sources | |
|---|---|
| Domains | Streaming, Fault_Tolerance |
| Last Updated | 2026-02-08 22:00 GMT |
Overview
Abstract base class representing a serializable handle to a record written in a WriteAheadLog.
Description
⚠️ DEPRECATED: DStream-based Spark Streaming is deprecated. Use Structured Streaming for new applications. See Heuristic:Apache_Spark_Warning_Deprecated_DStream_Streaming.
WriteAheadLogRecordHandle is a Java abstract class annotated with `@DeveloperApi` that implements `java.io.Serializable`. It represents an opaque handle returned by `WriteAheadLog.write()` that contains all information necessary to read back the written record. Concrete implementations must store the storage location or reference needed to retrieve the record. The handle must be serializable because it is stored in Spark's metadata for recovery purposes.
Usage
Extend this class when implementing a custom `WriteAheadLog`. The handle returned by your `write()` method must contain enough information (e.g., file path, offset, length) to locate and read the record via `WriteAheadLog.read()`.
Code Reference
Source Location
- Repository: Apache_Spark
- File: streaming/.../WriteAheadLogRecordHandle.java
- Lines: 1-32
Signature
@DeveloperApi
public abstract class WriteAheadLogRecordHandle implements java.io.Serializable {
}
Import
import org.apache.spark.streaming.util.WriteAheadLogRecordHandle;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| (implementation-specific) | — | — | Subclasses define their own fields (e.g., file path, offset) |
Outputs
| Name | Type | Description |
|---|---|---|
| Serialized handle | byte[] | Serializable representation for recovery metadata |
Usage Examples
Custom Record Handle
import org.apache.spark.streaming.util.WriteAheadLogRecordHandle;
public class MyRecordHandle extends WriteAheadLogRecordHandle {
private final String filePath;
private final long offset;
private final int length;
public MyRecordHandle(String filePath, long offset, int length) {
this.filePath = filePath;
this.offset = offset;
this.length = length;
}
public String getFilePath() { return filePath; }
public long getOffset() { return offset; }
public int getLength() { return length; }
}