Implementation:Lance format Lance LegacyValueEncoding

Knowledge Sources	Lance
Domains	Encoding, Legacy_Format
Last Updated	2026-02-08 19:33 GMT

Overview

The legacy value encoding is the simplest physical encoding that stores fixed-width values as contiguous byte buffers with optional compression in the Lance v2.0 format.

Description

⚠️ DEPRECATED: This is legacy code from the Lance v1/v2.0 format, retained only for backward compatibility. See Lance_format_Lance_Warning_Deprecated_Legacy_Encodings.

This module implements the flat value page encoding for the legacy (v2.0) Lance file format. ValuePageScheduler reads buffers of fixed-size items stored as-is on disk, computing byte ranges from row ranges using bytes_per_value. It supports optional compression via CompressionConfig: when compression is enabled (e.g., zstd), the entire page is read as a single range and decompressed on first access using GeneralBufferCompressor. Decompressed data is cached in an Arc<Mutex<Option<Vec<Bytes>>>> to avoid redundant decompressions. For uncompressed data, precise byte ranges are fetched directly. The ValuePageDecoder produces FixedWidthDataBlock output. The ValueEncoder on the encoding side is the default encoder that writes fixed-width data blocks as flat buffers with a protobuf Flat encoding descriptor.

Usage

Use this encoding for all fixed-width numeric and primitive data. ValuePageScheduler is the most commonly used physical scheduler, created by the physical dispatch for Flat encodings with bits_per_value that is a multiple of 8. ValueEncoder is the default array encoder used by BasicEncoder and is the building block for most other encodings.

Code Reference

Source Location

rust/lance-encoding/src/previous/encodings/physical/value.rs

Signature

#[derive(Debug, Clone, Copy)]
pub struct ValuePageScheduler {
    bytes_per_value: u64,
    buffer_offset: u64,
    buffer_size: u64,
    compression_config: CompressionConfig,
}

impl ValuePageScheduler {
    pub fn new(
        bytes_per_value: u64,
        buffer_offset: u64,
        buffer_size: u64,
        compression_config: CompressionConfig,
    ) -> Self;
}

impl PageScheduler for ValuePageScheduler { /* ... */ }

// ValueEncoder is defined in the main encodings module but used by legacy
pub struct ValueEncoder { /* default encoder */ }

Import

use lance_encoding::previous::encodings::physical::value::ValuePageScheduler;
use lance_encoding::encodings::physical::value::ValueEncoder;

I/O Contract

Input	Type	Description
bytes_per_value	u64	Number of bytes per value element
buffer_offset	u64	Starting position of the buffer in the file
buffer_size	u64	Total size of the buffer (needed for compressed pages)
compression_config	CompressionConfig	Compression scheme and level (None, Zstd, etc.)
ranges	&[Range<u64>]	Row ranges to decode

Output	Type	Description
decoded	DataBlock::FixedWidth	Fixed-width data block with raw value bytes
encoded	EncodedArray	Flat buffer with encoding descriptor

Usage Examples

use lance_encoding::previous::encodings::physical::value::ValuePageScheduler;
use lance_encoding::encodings::physical::block::{CompressionConfig, CompressionScheme};
use lance_encoding::decoder::PageScheduler;
use std::sync::Arc;

// Create an uncompressed value scheduler for 4-byte (Int32) values
let scheduler = ValuePageScheduler::new(
    4,                                              // bytes_per_value
    0,                                              // buffer_offset
    4000,                                           // buffer_size
    CompressionConfig::new(CompressionScheme::None, None),
);

// Create a zstd-compressed value scheduler
let compressed_scheduler = ValuePageScheduler::new(
    4,
    0,
    2000,  // compressed buffer size
    CompressionConfig::new(CompressionScheme::Zstd, Some(0)),
);

// Schedule ranges for reading
let ranges = vec![0..100];
let io: Arc<dyn EncodingsIo> = /* from context */;
let decoder_fut = scheduler.schedule_ranges(&ranges, &io, 0);

Related Pages

• Lance_format_Lance_LegacyPhysicalDispatch - Creates ValuePageScheduler from Flat protobuf encoding
• Lance_format_Lance_LegacyBasicEncoding - Wraps value encoding with nullability support
• Lance_format_Lance_LegacyBitmapEncoding - Alternative for 1-bit values (booleans)
• Lance_format_Lance_LegacyBitpackEncoding - Alternative for compressed integer values
• Lance_format_Lance_LegacyEncoder - Uses ValueEncoder as default array encoder
• Heuristic:Lance_format_Lance_Warning_Deprecated_Legacy_Encodings