Implementation:Lance format Lance LegacyBasicEncoding

Knowledge Sources	Lance
Domains	Encoding, Legacy_Format
Last Updated	2026-02-08 19:33 GMT

Overview

The legacy basic encoding is the foundational physical encoding that combines an optional validity bitmap with a values buffer to handle nullable data in the Lance v2.0 format.

Description

⚠️ DEPRECATED: This is legacy code from the Lance v1/v2.0 format, retained only for backward compatibility. See Lance_format_Lance_Warning_Deprecated_Legacy_Encodings.

This module implements the basic page encoding for the legacy (v2.0) Lance file format. BasicPageScheduler manages three null-status modes: no-nulls (values only, no validity bitmap), some-nulls (both validity bitmap and values buffers), and all-nulls (no buffers at all). Each mode delegates to appropriate inner PageScheduler instances for the validity and values buffers. The BasicPageDecoder mirrors this structure, combining decoded validity and values into NullableDataBlock, AllNullDataBlock, or plain data blocks depending on the mode. On the encoding side, BasicEncoder wraps an inner ArrayEncoder for values and produces protobuf encoding descriptors that capture the nullability pattern. This encoding is the top-level wrapper around virtually all primitive data in v2.0 files.

Usage

Use this encoding as the top-level wrapper for any primitive data that may contain null values. BasicPageScheduler is created by the physical dispatch when encountering Nullable protobuf encoding, which is the standard envelope for primitive column pages. On the encoding side, BasicEncoder wraps value encoders like ValueEncoder or BitpackedForNonNegArrayEncoder.

Code Reference

Source Location

rust/lance-encoding/src/previous/encodings/physical/basic.rs

Signature

pub struct BasicPageScheduler {
    mode: SchedulerNullStatus,
}

impl BasicPageScheduler {
    pub fn new_nullable(
        validity_decoder: Box<dyn PageScheduler>,
        values_decoder: Box<dyn PageScheduler>,
    ) -> Self;
    pub fn new_non_nullable(values_decoder: Box<dyn PageScheduler>) -> Self;
    pub fn new_all_null() -> Self;
}

pub struct BasicEncoder { /* fields omitted */ }

impl BasicEncoder {
    pub fn new(values_encoder: Box<dyn ArrayEncoder>) -> Self;
}

Import

use lance_encoding::previous::encodings::physical::basic::{
    BasicPageScheduler, BasicEncoder,
};

I/O Contract

Input	Type	Description
validity_decoder	Box<dyn PageScheduler>	Scheduler for the validity bitmap (optional)
values_decoder	Box<dyn PageScheduler>	Scheduler for the values buffer (optional for all-null)
data	DataBlock	Nullable, all-null, or plain data block to encode

Output	Type	Description
decoded	DataBlock	NullableDataBlock, AllNullDataBlock, or plain DataBlock
encoded	EncodedArray	Values and optional validity with encoding descriptor

Usage Examples

use lance_encoding::previous::encodings::physical::basic::{BasicPageScheduler, BasicEncoder};
use lance_encoding::decoder::PageScheduler;

// Create a nullable basic page scheduler
let validity_scheduler: Box<dyn PageScheduler> = /* bitmap scheduler */;
let values_scheduler: Box<dyn PageScheduler> = /* value scheduler */;
let scheduler = BasicPageScheduler::new_nullable(validity_scheduler, values_scheduler);

// Create a non-nullable scheduler (no validity bitmap)
let values_scheduler: Box<dyn PageScheduler> = /* value scheduler */;
let scheduler = BasicPageScheduler::new_non_nullable(values_scheduler);

// Create an all-null scheduler
let scheduler = BasicPageScheduler::new_all_null();

Related Pages

• Lance_format_Lance_LegacyPhysicalDispatch - Creates BasicPageScheduler from Nullable protobuf
• Lance_format_Lance_LegacyBitmapEncoding - Used for the validity bitmap buffer
• Lance_format_Lance_LegacyValueEncoding - Used for the values buffer
• Lance_format_Lance_LegacyBitpackEncoding - Alternative values encoder
• Lance_format_Lance_LegacyPrimitiveEncoding - Logical encoding built on top of basic
• Heuristic:Lance_format_Lance_Warning_Deprecated_Legacy_Encodings