Implementation:Lance format Lance LegacyPhysicalDispatch

Knowledge Sources	Lance
Domains	Encoding, Legacy_Format
Last Updated	2026-02-08 19:33 GMT

Overview

The legacy physical dispatch module converts protobuf-encoded array descriptions into concrete physical page schedulers for the Lance v2.0 format.

Description

⚠️ DEPRECATED: This is legacy code from the Lance v1/v2.0 format, retained only for backward compatibility. See Lance_format_Lance_Warning_Deprecated_Legacy_Encodings.

This module is the central dispatch point for the legacy (v2.0) Lance file format decoding pipeline. The decoder_from_array_encoding function takes a protobuf ArrayEncoding message and recursively constructs the appropriate PageScheduler tree. It handles all v2.0 encoding variants: Nullable (dispatching to BasicPageScheduler with no-nulls, some-nulls, or all-nulls modes), Flat (dispatching to ValuePageScheduler or DenseBitmapScheduler based on bits-per-value), FixedSizeList, List (offsets), Binary, Fsst, Dictionary, FixedSizeBinary, PackedStruct, Bitpacked, and BitpackedForNonNeg. The get_buffer helper translates protobuf buffer descriptors into file positions using page, column, or file buffer types. The get_buffer_decoder helper creates value or bitmap schedulers from flat encoding descriptors with optional compression support.

Usage

Use this module when building the decoding pipeline from Lance v2.0 file metadata. decoder_from_array_encoding is called by PrimitiveFieldScheduler to create page-level schedulers from the protobuf encoding stored in page metadata. It is the bridge between serialized file metadata and the runtime decoding infrastructure.

Code Reference

Source Location

rust/lance-encoding/src/previous/encodings/physical.rs

Signature

pub fn decoder_from_array_encoding(
    encoding: &pb::ArrayEncoding,
    buffers: &PageBuffers,
    data_type: &DataType,
) -> Box<dyn PageScheduler>;

fn get_buffer(buffer_desc: &pb::Buffer, buffers: &PageBuffers) -> (u64, u64);

fn get_buffer_decoder(
    encoding: &pb::Flat,
    buffers: &PageBuffers,
) -> Box<dyn PageScheduler>;

Import

use lance_encoding::previous::encodings::physical::decoder_from_array_encoding;

I/O Contract

Input	Type	Description
encoding	&pb::ArrayEncoding	Protobuf array encoding descriptor from page metadata
buffers	&PageBuffers	Buffer position mappings (page, column, and file level)
data_type	&DataType	Arrow data type for the column being decoded

Output	Type	Description
scheduler	Box<dyn PageScheduler>	Concrete page scheduler for the specified encoding

Usage Examples

use lance_encoding::previous::encodings::physical::decoder_from_array_encoding;
use lance_encoding::decoder::{PageBuffers, ColumnBuffers, FileBuffers};
use lance_encoding::format::pb;
use arrow_schema::DataType;

// Given protobuf encoding from file metadata
let encoding: pb::ArrayEncoding = /* from page info */;
let buffers = PageBuffers {
    column_buffers: ColumnBuffers {
        file_buffers: FileBuffers {
            positions_and_sizes: &[(0, 4096)],
        },
        positions_and_sizes: &[(4096, 1024)],
    },
    positions_and_sizes: &[(5120, 512)],
};

// Create a page scheduler from the encoding
let scheduler = decoder_from_array_encoding(
    &encoding,
    &buffers,
    &DataType::Int32,
);

Related Pages

• Lance_format_Lance_LegacyPrimitiveEncoding - Calls this dispatch to build page schedulers
• Lance_format_Lance_LegacyBasicEncoding - BasicPageScheduler created by Nullable dispatch
• Lance_format_Lance_LegacyValueEncoding - ValuePageScheduler created by Flat dispatch
• Lance_format_Lance_LegacyBitmapEncoding - DenseBitmapScheduler created for 1-bit values
• Lance_format_Lance_LegacyBinaryEncoding - BinaryPageScheduler created by Binary dispatch
• Lance_format_Lance_LegacyDictionaryEncoding - DictionaryPageScheduler created by Dictionary dispatch
• Lance_format_Lance_LegacyFsstEncoding - FsstPageScheduler created by Fsst dispatch
• Lance_format_Lance_LegacyBitpackEncoding - BitpackedScheduler created by Bitpacked dispatch
• Lance_format_Lance_LegacyFixedSizeBinaryEncoding - FixedSizeBinaryPageScheduler dispatch
• Lance_format_Lance_LegacyPackedStructEncoding - PackedStructPageScheduler dispatch
• Lance_format_Lance_LegacyFixedSizeListEncoding - FixedListScheduler dispatch
• Heuristic:Lance_format_Lance_Warning_Deprecated_Legacy_Encodings