Implementation:Lance format Lance Java CompactionTask

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

Description

CompactionTask is a serializable class representing a single unit of work in a distributed compaction pipeline. Each task encapsulates a TaskData object (containing the list of fragment identifiers to rewrite), the dataset read version at planning time, and the CompactionOptions used during planning. The execute() method runs the compaction rewrite against a given Dataset and returns a RewriteResult, delegating to the native Rust engine via JNI.

Usage

CompactionTask instances are produced by Compaction.planCompaction() as part of a CompactionPlan. Because CompactionTask implements Serializable, tasks can be distributed across network boundaries (e.g., Spark executors). Each task is executed independently, and the resulting RewriteResult objects are collected and passed to Compaction.commitCompaction() to finalize the compaction.

Code Reference

Source Location

java/src/main/java/org/lance/compaction/CompactionTask.java

Signature

public class CompactionTask implements Serializable {
    public CompactionTask(TaskData taskData, long readVersion,
                          CompactionOptions compactionOptions);
    public RewriteResult execute(Dataset dataset);
    public CompactionOptions getCompactionOptions();
    public long getReadVersion();
    public TaskData getTaskData();
    public String toString();
}

Import

import org.lance.compaction.CompactionTask;

I/O Contract

Constructor Parameters

Name	Type	Description
taskData	TaskData	Contains the list of fragments to be rewritten
readVersion	long	The dataset version at the time of planning
compactionOptions	CompactionOptions	Compaction configuration for this task

execute() Parameters

Name	Type	Description
dataset	Dataset	The dataset to execute the compaction against

Return Values

Method	Return Type	Description
execute()	RewriteResult	The result of the compaction rewrite operation
getCompactionOptions()	CompactionOptions	Returns the compaction options for this task
getReadVersion()	long	Returns the read version used for this task
getTaskData()	TaskData	Returns the task data containing fragment information

Usage Examples

import org.lance.compaction.Compaction;
import org.lance.compaction.CompactionOptions;
import org.lance.compaction.CompactionPlan;
import org.lance.compaction.CompactionTask;
import org.lance.compaction.RewriteResult;
import org.lance.Dataset;
import java.util.ArrayList;
import java.util.List;

// Plan compaction to get tasks
CompactionOptions options = CompactionOptions.builder()
    .withTargetRowsPerFragment(1024 * 1024)
    .build();

Dataset dataset = Dataset.open().uri("/data/dataset.lance").build();
CompactionPlan plan = Compaction.planCompaction(dataset, options);

// Execute tasks sequentially
List<RewriteResult> results = new ArrayList<>();
for (CompactionTask task : plan.getTasks()) {
    System.out.println("Executing task: " + task);
    System.out.println("Read version: " + task.getReadVersion());
    System.out.println("Fragments: " + task.getTaskData().getFragments());

    RewriteResult result = task.execute(dataset);
    results.add(result);
}

// In a distributed environment, tasks can be serialized and sent to workers:
// ObjectOutputStream out = new ObjectOutputStream(networkStream);
// out.writeObject(task);  // CompactionTask is Serializable

// Commit all results
Compaction.commitCompaction(dataset, results, options);

Related Pages

• Lance_format_Lance_Java_Compaction - Creates CompactionTask objects via planCompaction() and commits results
• Lance_format_Lance_Java_CompactionOptions - Configuration carried by each CompactionTask
• Lance_format_Lance_Java_FragmentMetadata - Fragment metadata that TaskData references