Implementation:Lance format Lance Java Compaction

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

Description

Compaction is a static utility class that serves as the entrypoint for distributed compaction operations on Lance datasets. It provides two main operations: planCompaction (which analyzes a dataset and creates a plan of rewrite tasks) and commitCompaction (which commits completed rewrite results back to the dataset). Both methods delegate to native JNI methods that invoke the Rust compaction engine. The class loads the native library via JniLoader.ensureLoaded() in its static initializer.

Usage

Compaction follows a distributed execution pattern: first call planCompaction() to get a CompactionPlan containing CompactionTask objects, execute the tasks (potentially in parallel on distributed workers), collect the RewriteResult outputs, and finally call commitCompaction() to atomically apply all changes. Both methods require non-null inputs and validate with Guava Preconditions.

Code Reference

Source Location

java/src/main/java/org/lance/compaction/Compaction.java

Signature

public class Compaction {
    public static CompactionPlan planCompaction(
        Dataset dataset, CompactionOptions compactionOptions);
    public static CompactionMetrics commitCompaction(
        Dataset dataset, List<RewriteResult> rewriteResults,
        CompactionOptions compactionOptions);
}

Import

import org.lance.compaction.Compaction;

I/O Contract

planCompaction Parameters

Name	Type	Description
dataset	Dataset	The dataset to analyze for compaction (must not be null)
compactionOptions	CompactionOptions	Configuration controlling compaction behavior (must not be null)

commitCompaction Parameters

Name	Type	Description
dataset	Dataset	The dataset to commit compaction results to (must not be null)
rewriteResults	List<RewriteResult>	Results from executing compaction tasks (must not be null)
compactionOptions	CompactionOptions	Same options used during planning (must not be null)

Return Values

Method	Return Type	Description
planCompaction()	CompactionPlan	A plan containing CompactionTask objects to be executed
commitCompaction()	CompactionMetrics	Metrics describing the committed compaction (fragments rewritten, etc.)

Usage Examples

import org.lance.compaction.Compaction;
import org.lance.compaction.CompactionOptions;
import org.lance.compaction.CompactionPlan;
import org.lance.compaction.CompactionTask;
import org.lance.compaction.CompactionMetrics;
import org.lance.compaction.RewriteResult;
import org.lance.Dataset;
import java.util.ArrayList;
import java.util.List;

// Configure compaction options
CompactionOptions options = CompactionOptions.builder()
    .withTargetRowsPerFragment(1024 * 1024)
    .withMaxRowsPerGroup(1024)
    .build();

// Plan compaction
Dataset dataset = Dataset.open().uri("/data/dataset.lance").build();
CompactionPlan plan = Compaction.planCompaction(dataset, options);

// Execute tasks (can be distributed)
List<RewriteResult> results = new ArrayList<>();
for (CompactionTask task : plan.getTasks()) {
    RewriteResult result = task.execute(dataset);
    results.add(result);
}

// Commit all results atomically
CompactionMetrics metrics = Compaction.commitCompaction(dataset, results, options);

Related Pages

• Lance_format_Lance_Java_CompactionOptions - Configuration passed to both planCompaction and commitCompaction
• Lance_format_Lance_Java_CompactionTask - Individual tasks produced by planCompaction and executed on workers
• Lance_format_Lance_Java_CleanupPolicy - Cleanup policy for removing old versions after compaction
• Lance_format_Lance_Java_JniLoader - Loads the native library used by Compaction