Implementation:Lance format Lance JNI BlockingDataset

Knowledge Sources	Lance
Domains	Java_Bindings, JNI
Last Updated	2026-02-08 19:33 GMT

Overview

JNI BlockingDataset is the Rust-side JNI binding that wraps the Lance Dataset type, exposing dataset lifecycle operations (open, write, commit, version management, scanning, indexing, and deletion) to Java through synchronous blocking calls on a shared Tokio runtime.

Description

The BlockingDataset struct holds an inner lance::dataset::Dataset and provides synchronous wrappers around its asynchronous methods by executing futures on a global Tokio runtime (RT). It supports the complete dataset lifecycle:

• Opening and writing datasets with configurable storage options, read parameters, and optional serialized manifests.
• Version management including listing versions, checking out specific versions or tags, creating/deleting/updating tags, and restoring previous versions.
• Branch management including listing branches, deleting branches, and checking out references.
• Data operations such as counting rows, calculating data statistics, adding columns, altering columns, dropping columns, merging datasets, deleting rows by filter, and updating rows.
• Index management including creating vector and scalar indices, listing indices, and optimizing indices.
• Dataset maintenance including compaction, cleanup of old versions, and dropping entire datasets.
• Schema and transaction operations for reading the Lance schema, listing fragments, and committing transactions.
• Storage option management for retrieving initial and refreshed storage credentials.

Each JNI-exported function follows the pattern of receiving Java objects, converting them to Rust types using the JNIEnvExt and FromJObjectWithEnv traits, performing the operation, and converting results back to Java objects via the IntoJava trait.

Usage

Use this module when building or maintaining the Java SDK for Lance. It is the primary bridge between the Java Dataset class and the native Rust implementation. Java applications call methods on the Java Dataset object, which delegates to these JNI functions through native method declarations.

Code Reference

Source Location

java/lance-jni/src/blocking_dataset.rs

Signature

pub struct BlockingDataset {
    pub(crate) inner: Dataset,
}

impl BlockingDataset {
    pub fn open(uri: &str, version: Option<u64>, block_size: Option<i32>,
        index_cache_size_bytes: i64, metadata_cache_size_bytes: i64,
        storage_options: HashMap<String, String>,
        serialized_manifest: Option<&[u8]>,
        storage_options_provider: Option<Arc<dyn StorageOptionsProvider>>,
    ) -> Result<Self>;

    pub fn write(reader: impl RecordBatchReader + Send + 'static,
        uri: &str, params: Option<WriteParams>) -> Result<Self>;

    pub fn commit(uri: &str, operation: Operation,
        read_version: Option<u64>,
        storage_options: HashMap<String, String>) -> Result<Self>;

    pub fn count_rows(&self, filter: Option<String>) -> Result<usize>;
    pub fn latest_version(&self) -> Result<u64>;
    pub fn list_versions(&self) -> Result<Vec<Version>>;
    pub fn version(&self) -> Result<Version>;
    pub fn checkout_version(&mut self, version: u64) -> Result<Self>;
    pub fn drop(uri: &str, storage_options: HashMap<String, String>) -> Result<()>;
    // ... additional methods
}

Import

use crate::blocking_dataset::{BlockingDataset, NATIVE_DATASET};

I/O Contract

Direction	Type	Description
Input	JObject (Java Dataset)	Java object carrying the native dataset handle
Input	JString (URI)	Dataset URI (local path or object store URL)
Input	JObject (storage options Map)	Java Map<String, String> for storage configuration
Input	jlong (arrow stream address)	Memory address of an Arrow FFI stream for write operations
Output	JObject (Java Dataset)	Java Dataset object with native handle attached
Output	jint / jlong	Scalar results such as row counts or version numbers
Output	jbyteArray	Serialized data such as schema bytes or IPC-encoded record batches

Usage Examples

// Java side: opening a dataset
import org.lance.Dataset;

Map<String, String> storageOptions = new HashMap<>();
storageOptions.put("region", "us-east-1");

Dataset dataset = Dataset.open("s3://my-bucket/my-dataset", storageOptions);
long rowCount = dataset.countRows();
int latestVersion = dataset.latestVersion();

// Rust JNI side: how the native open function is structured
#[no_mangle]
pub extern "system" fn Java_org_lance_Dataset_openNative<'local>(
    mut env: JNIEnv<'local>,
    _obj: JObject,
    uri: JString,
    version: JObject,
    // ... additional parameters
) -> JObject<'local> {
    ok_or_throw!(env, inner_open(&mut env, uri, version, /* ... */))
}

Related Pages

• Lance_format_Lance_JNI_BlockingScanner - Scanner created from BlockingDataset for reading data
• Lance_format_Lance_JNI_Fragment - Fragment operations used within dataset write and compaction
• Lance_format_Lance_JNI_Transaction - Transaction types committed through BlockingDataset
• Lance_format_Lance_JNI_Utils - Utility functions for parameter extraction used by BlockingDataset
• Lance_format_Lance_JNI_FFI - JNIEnvExt trait used for Java-to-Rust type conversions
• Lance_format_Lance_JNI_Schema - Schema conversion used when reading dataset schemas