Implementation:Lance format Lance Java OverwriteOp

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

Description

The Overwrite class is an immutable operation that replaces the entire content of an existing Lance dataset with new fragments and a new schema. It extends SchemaOperation (which implements Operation) and includes an optional map of table configuration key-value pairs to upsert during the overwrite.

The operation carries a list of FragmentMetadata, an Arrow Schema, and optional configUpsertValues. Notably, config keys not present in the upsert map are preserved (not deleted). For creating entirely new datasets, the Dataset.create method should be used instead.

Usage

Use Overwrite when the entire dataset needs to be replaced with new data and potentially a new schema, while retaining the same dataset URI. This is appropriate for periodic full refreshes, schema migrations, or rebuilding a dataset from scratch. The optional config upsert values allow updating table-level configuration in the same transaction.

Code Reference

Source Location

java/src/main/java/org/lance/operation/Overwrite.java

Signature

public class Overwrite extends SchemaOperation {
    public static Builder builder();
    public List<FragmentMetadata> fragments();
    public Optional<Map<String, String>> configUpsertValues();
    public Schema schema();       // inherited from SchemaOperation
    public long exportSchema(BufferAllocator allocator); // inherited
    public String name();         // returns "Overwrite"
}

Import

import org.lance.operation.Overwrite;

I/O Contract

Inputs
Parameter	Type	Required	Description
fragments	`List<FragmentMetadata>`	Yes	Fragment metadata for the replacement data
schema	`org.apache.arrow.vector.types.pojo.Schema`	Yes	The new Arrow schema for the dataset
configUpsertValues	`Map<String, String>`	No	Table configuration key-value pairs to upsert (null means no config changes)

Outputs
Return	Type	Description
fragments()	`List<FragmentMetadata>`	The replacement fragment metadata
configUpsertValues()	`Optional<Map<String, String>>`	The table config values to upsert, if any
schema()	`Schema`	The new schema
name()	`String`	Returns `"Overwrite"` for JNI dispatch

Usage Examples

// Overwrite dataset with new data, schema, and config
Schema newSchema = new Schema(List.of(
    Field.nullable("id", new ArrowType.Int(64, true)),
    Field.nullable("value", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE))
));

Overwrite overwriteOp = Overwrite.builder()
    .fragments(newFragments)
    .schema(newSchema)
    .configUpsertValues(Map.of("lance.compaction.target_rows", "1000000"))
    .build();

String opName = overwriteOp.name(); // "Overwrite"

Related Pages

Lance_format_Lance_Java_AppendOp -- Appends new data without replacing existing content
Lance_format_Lance_Java_MergeOp -- Merges new columns into an existing dataset
Lance_format_Lance_Java_DeleteOp -- Removes specific rows by predicate

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment