Implementation:Lance format Lance Java DirectoryNamespace

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

The DirectoryNamespace class provides a directory-based implementation of the LanceNamespace interface, enabling namespace and table management operations against local filesystem, S3, Azure Blob Storage, and Google Cloud Storage backends.

Description

DirectoryNamespace implements LanceNamespace and Closeable, wrapping a native Rust implementation via JNI. It serializes all requests and responses as JSON for communication with the native layer. The class supports:

Storage backends:

• Local filesystem (e.g., /path/to/dir)
• AWS S3 (e.g., s3://bucket/path)
• Azure Blob Storage (e.g., az://container/path)
• Google Cloud Storage (e.g., gs://bucket/path)

Namespace operations:

• listNamespaces, describeNamespace, createNamespace, dropNamespace, namespaceExists

Table operations:

• listTables, describeTable, createTable, createEmptyTable, declareTable
• registerTable, deregisterTable, dropTable, tableExists
• insertIntoTable, mergeInsertIntoTable, updateTable, deleteFromTable
• queryTable, countTableRows

Index operations:

• createTableIndex, listTableIndices, describeTableIndexStats

Transaction operations:

• describeTransaction, alterTransaction

Configuration properties:

• root (required): Root directory path or URI
• manifest_enabled: Enable manifest tracking (default: true)
• dir_listing_enabled: Enable directory listing (default: true)
• inline_optimization_enabled: Enable inline optimizations (default: true)
• storage.*: Cloud provider storage options

Credential vending:

• credential_vendor.enabled: Enable dynamic credential vending
• AWS: credential_vendor.aws_role_arn, aws_external_id, aws_region, etc.
• GCP: credential_vendor.gcp_service_account
• Azure: credential_vendor.azure_account_name, azure_tenant_id, etc.

Dynamic context provider:

• Supports DynamicContextProvider for per-request context (e.g., auth headers)
• Can be loaded from classpath via dynamic_context_provider.impl property

Usage

Use DirectoryNamespace when you need a catalog/namespace layer for organizing Lance tables in directory-based storage. It is commonly used with WriteDatasetBuilder and OpenDatasetBuilder for namespace-managed dataset access with automatic credential vending.

Code Reference

Source Location

Property	Value
File	java/src/main/java/org/lance/namespace/DirectoryNamespace.java
Package	org.lance.namespace
Lines	567

Signature

public class DirectoryNamespace implements LanceNamespace, Closeable

Import

import org.lance.namespace.DirectoryNamespace;

I/O Contract

Lifecycle Methods

Method	Input	Output	Description
DirectoryNamespace()	none	instance	Creates uninitialized namespace
initialize(Map, BufferAllocator)	properties, allocator	void	Initializes with config properties
initialize(Map, BufferAllocator, DynamicContextProvider)	properties, allocator, provider	void	Initializes with dynamic context provider
close()	none	void	Releases native resources

Key Namespace Operations

Method	Input	Output	Description
namespaceId()	none	String	Returns the namespace identifier
listNamespaces(ListNamespacesRequest)	request	ListNamespacesResponse	Lists child namespaces
createNamespace(CreateNamespaceRequest)	request	CreateNamespaceResponse	Creates a namespace
dropNamespace(DropNamespaceRequest)	request	DropNamespaceResponse	Drops a namespace

Key Table Operations

Method	Input	Output	Description
listTables(ListTablesRequest)	request	ListTablesResponse	Lists tables
describeTable(DescribeTableRequest)	request	DescribeTableResponse	Describes a table with location and credentials
declareTable(DeclareTableRequest)	request	DeclareTableResponse	Declares a table location without creating data
createTable(CreateTableRequest, byte[])	request, data	CreateTableResponse	Creates a table with initial data
dropTable(DropTableRequest)	request	DropTableResponse	Drops a table
queryTable(QueryTableRequest)	request	byte[]	Queries table data as bytes

Usage Examples

Local Filesystem Namespace

import org.lance.namespace.DirectoryNamespace;
import org.lance.namespace.model.*;
import org.apache.arrow.memory.RootAllocator;
import java.util.HashMap;
import java.util.Map;

Map<String, String> properties = new HashMap<>();
properties.put("root", "/tmp/lance-data");
properties.put("manifest_enabled", "true");

DirectoryNamespace namespace = new DirectoryNamespace();
namespace.initialize(properties, new RootAllocator(Long.MAX_VALUE));

// List tables
ListTablesRequest request = new ListTablesRequest();
ListTablesResponse response = namespace.listTables(request);

namespace.close();

S3 Namespace

import org.lance.namespace.DirectoryNamespace;
import java.util.HashMap;
import java.util.Map;

Map<String, String> properties = new HashMap<>();
properties.put("root", "s3://my-bucket/lance-data");
properties.put("storage.region", "us-east-1");

DirectoryNamespace namespace = new DirectoryNamespace();
namespace.initialize(properties, allocator);
// Use namespace for table operations...
namespace.close();

S3 with Credential Vending

import org.lance.namespace.DirectoryNamespace;
import java.util.HashMap;
import java.util.Map;

Map<String, String> properties = new HashMap<>();
properties.put("root", "s3://my-bucket/lance-data");
properties.put("credential_vendor.enabled", "true");
properties.put("credential_vendor.aws_role_arn",
    "arn:aws:iam::123456789012:role/MyRole");
properties.put("credential_vendor.aws_duration_millis", "3600000");

DirectoryNamespace namespace = new DirectoryNamespace();
namespace.initialize(properties, allocator);
// describeTable() now returns vended credentials
namespace.close();

Using with WriteDatasetBuilder

import org.lance.Dataset;
import org.lance.WriteParams;
import java.util.Arrays;

Dataset dataset = Dataset.write()
    .reader(myArrowReader)
    .namespace(namespace)
    .tableId(Arrays.asList("my_table"))
    .mode(WriteParams.WriteMode.CREATE)
    .execute();

Related Pages

• Lance_format_Lance_Java_RestNamespace - REST API-based namespace implementation
• Lance_format_Lance_Java_Dataset - Dataset class that integrates with namespaces
• Lance_format_Lance_Java_WriteDatasetBuilder - Builder that uses namespaces for write operations