Implementation:Lance format Lance SessionCaches

Knowledge Sources	Lance
Domains	Core, Caching
Last Updated	2026-02-08 19:33 GMT

Overview

Description

The SessionCaches module provides a hierarchical caching system for Lance dataset metadata. The caching hierarchy is designed to prevent key collisions across datasets and files:

GlobalMetadataCache
  |
  +-- DSMetadataCache (prefixed by dataset URI)
       |
       +-- FileMetadataCache (prefixed by file path)

The module defines the following types:

• GlobalMetadataCache -- Top-level cache wrapper around LanceCache. Provides for_dataset(uri) to create dataset-scoped sub-caches and file_metadata_cache(path) for file-level caching.
• DSMetadataCache -- Dataset-scoped cache that derefs to LanceCache. Provides file_metadata_cache(prefix) for further file-level scoping.
• Type-safe cache keys -- Six key types that implement CacheKey with specific value types:
  • ManifestKey -- Caches Manifest objects, keyed by version and optional e-tag
  • TransactionKey -- Caches Transaction objects, keyed by version
  • DeletionFileKey -- Caches DeletionVector objects, keyed by fragment ID and deletion file metadata
  • RowAddrMaskKey -- Caches RowAddrMask objects, keyed by version
  • RowIdIndexKey -- Caches RowIdIndex objects, keyed by version
  • RowIdSequenceKey -- Caches RowIdSequence objects, keyed by fragment ID

Usage

The session cache is instantiated as part of a Lance Session and shared across all dataset operations within that session. It accelerates repeated access to manifests, deletion vectors, and row ID structures.

Code Reference

Source Location

rust/lance/src/session/caches.rs

Signature

pub struct GlobalMetadataCache(pub(super) LanceCache);

impl GlobalMetadataCache {
    pub fn for_dataset(&self, uri: &str) -> DSMetadataCache;
    pub(crate) fn file_metadata_cache(&self, path: &Path) -> LanceCache;
}

pub struct DSMetadataCache(pub(crate) LanceCache);

impl DSMetadataCache {
    pub(crate) fn file_metadata_cache(&self, prefix: &Path) -> LanceCache;
}

// Cache key examples
pub struct ManifestKey<'a> { pub version: u64, pub e_tag: Option<&'a str> }
pub struct TransactionKey { pub version: u64 }
pub struct DeletionFileKey<'a> { pub fragment_id: u64, pub deletion_file: &'a DeletionFile }
pub struct RowAddrMaskKey { pub version: u64 }
pub struct RowIdIndexKey { pub version: u64 }
pub struct RowIdSequenceKey { pub fragment_id: u64 }

Import

use lance::session::caches::{
    GlobalMetadataCache, DSMetadataCache,
    ManifestKey, TransactionKey, DeletionFileKey,
    RowAddrMaskKey, RowIdIndexKey, RowIdSequenceKey,
};

I/O Contract

Inputs

Parameter	Type	Description
uri	&str	Dataset URI used as a key prefix for namespace isolation
path	&Path	Object store path used as key prefix for file-level caching
version	u64	Dataset version number used in manifest, transaction, and mask cache keys
fragment_id	u64	Fragment identifier used in deletion and row ID sequence cache keys
e_tag	Option<&str>	Optional HTTP e-tag for manifest cache invalidation

Outputs

Type	Description
DSMetadataCache	A dataset-scoped sub-cache with URI prefix
LanceCache	A file-scoped sub-cache with path prefix
Cow<str>	Formatted cache key string (from CacheKey::key())

Usage Examples

use lance::session::caches::{GlobalMetadataCache, ManifestKey};
use lance_core::cache::LanceCache;

// Create a global cache and scope it to a dataset
let global_cache = GlobalMetadataCache(LanceCache::new(128 * 1024 * 1024));
let ds_cache = global_cache.for_dataset("s3://bucket/my_dataset.lance");

// Look up a cached manifest
let key = ManifestKey { version: 42, e_tag: None };
// ds_cache.get(&key) -> Option<Arc<Manifest>>

Related Pages

• Lance_format_Lance_IndexCaches -- Parallel caching hierarchy for index data
• Lance_format_Lance_CrateRoot -- Main crate containing the session module