Implementation:Apache Flink WritableTypeInfo
| Knowledge Sources | |
|---|---|
| Domains | Hadoop_Compatibility, Type_System |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
WritableTypeInfo provides Flink type information for data types that extend Hadoop's Writable interface, enabling serialization, deserialization, and comparison of Hadoop-compatible types within Flink's type system.
Description
WritableTypeInfo is a generic class parameterized by a type T that extends Hadoop's Writable interface. It extends Flink's TypeInformation<T> and implements the AtomicType<T> interface, meaning it represents a single, non-composite data type.
The class serves as the bridge between Hadoop's Writable-based type system and Flink's native type system. When a Flink program needs to work with Hadoop Writable types, WritableTypeInfo provides the metadata that Flink requires to serialize, deserialize, and compare values of those types.
Key behaviors include:
- Constructor validation: The constructor enforces that the provided class is a proper subclass of Writable (not the Writable interface itself).
- Serializer creation: The createSerializer method returns a WritableSerializer<T> instance for the given type class.
- Comparator creation: The createComparator method returns a WritableComparator if the type also implements Comparable; otherwise, it throws an UnsupportedOperationException.
- Key type support: The isKeyType method returns true only if the Writable subclass also implements Comparable, which is required for key-based operations such as grouping or joining.
- Type characteristics: The type reports itself as non-basic, non-tuple, with an arity and total field count of 1.
- Static factory method: The getWritableTypeInfo static method provides a validated factory for creating WritableTypeInfo instances, throwing InvalidTypesException for invalid classes.
Usage
Use WritableTypeInfo when you need to integrate Hadoop Writable types into a Flink pipeline. This is necessary when:
- Reading data from Hadoop InputFormats that produce Writable key-value pairs.
- Using Hadoop Writable types as intermediate data representations in Flink programs.
- Registering type information for custom Hadoop data types so Flink can properly serialize and compare them.
Code Reference
Source Location
- Repository: Apache_Flink
- File: flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/typeutils/WritableTypeInfo.java
- Lines: 1-159
Signature
@Public
public class WritableTypeInfo<T extends Writable> extends TypeInformation<T>
implements AtomicType<T>
Import
import org.apache.flink.api.java.typeutils.WritableTypeInfo;
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| typeClass | Class<T extends Writable> | Yes | The class of the Hadoop Writable subtype to represent in Flink's type system |
Outputs
| Name | Type | Description |
|---|---|---|
| TypeSerializer<T> | WritableSerializer<T> | Serializer for the Writable type, created via createSerializer |
| TypeComparator<T> | WritableComparator<T> | Comparator for the Writable type (only if T implements Comparable), created via createComparator |
| TypeInformation<T> | WritableTypeInfo<T> | Type information instance created via getWritableTypeInfo static factory method |
Usage Examples
// Create type information for a Hadoop Text Writable type
WritableTypeInfo<Text> textTypeInfo = new WritableTypeInfo<>(Text.class);
// Retrieve the serializer for use in Flink serialization
TypeSerializer<Text> serializer = textTypeInfo.createSerializer(serializerConfig);
// Check if the type can be used as a key (requires Comparable)
boolean canBeKey = textTypeInfo.isKeyType();
// Create a comparator for sorting (ascending order)
if (canBeKey) {
TypeComparator<Text> comparator = textTypeInfo.createComparator(true, executionConfig);
}
// Use the static factory method
TypeInformation<Text> typeInfo = WritableTypeInfo.getWritableTypeInfo(Text.class);