Implementation:Lance format Lance Java DataReplacementOp

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

Description

The DataReplacement class is an immutable operation that replaces column data within existing dataset fragments. It implements the Operation interface and is designed for null column population, where an entirely null column is replaced with a column containing actual data.

The operation enforces strict schema constraints: replacement files must contain the exact same schema (same set of columns) as the originals. It also requires that the affected columns share the same data file layout across all fragments being replaced. The class contains an inner DataReplacementGroup class that pairs a fragment ID with the replacement DataFile.

Usage

Use DataReplacement for late materialization scenarios where columns are initially written as null placeholders and later populated with computed or fetched data. This is common in pipelines where embedding columns are filled asynchronously after the initial data ingest.

Code Reference

Source Location

java/src/main/java/org/lance/operation/DataReplacement.java

Signature

public class DataReplacement implements Operation {
    public static Builder builder();
    public List<DataReplacementGroup> replacements();
    public String name();  // returns "DataReplacement"

    public static class DataReplacementGroup {
        public DataReplacementGroup(long fragmentId, DataFile replacedFile);
        public long fragmentId();
        public DataFile replacedFile();
    }
}

Import

import org.lance.operation.DataReplacement;
import org.lance.operation.DataReplacement.DataReplacementGroup;

I/O Contract

Inputs

Parameter	Type	Required	Description
replacements	List<DataReplacementGroup>	Yes	List of fragment ID and replacement data file pairs

DataReplacementGroup Fields

Field	Type	Description
fragmentId	long	The ID of the fragment whose data file is being replaced
replacedFile	DataFile	The new data file that replaces the old file in the fragment

Outputs

Return	Type	Description
replacements()	List<DataReplacementGroup>	The list of replacement groups
name()	String	Returns "DataReplacement" for JNI dispatch

Usage Examples

// Replace null column data in specific fragments
DataFile newDataFile = writeColumnData(embeddingValues);

DataReplacement.DataReplacementGroup group =
    new DataReplacement.DataReplacementGroup(fragmentId, newDataFile);

DataReplacement replaceOp = DataReplacement.builder()
    .replacements(List.of(group))
    .build();

String opName = replaceOp.name(); // "DataReplacement"

Related Pages

• Lance_format_Lance_Java_UpdateOp -- Updates rows or columns with new values
• Lance_format_Lance_Java_RewriteOp -- Rewrites data for compaction without changing content
• Lance_format_Lance_Java_MergeOp -- Merges new fragments with schema evolution