Implementation:Lance format Lance ConstantEncoding

Knowledge Sources	Lance
Domains	Encoding, Compression
Last Updated	2026-02-08 19:33 GMT

Overview

ConstantPageScheduler and encode_constant_page handle pages where every row has the same scalar value, storing the value once and expanding it at decode time.

Description

Constant encoding optimizes the case where an entire page contains a single repeated value. The scalar value can be stored either inline in the encoding description (for small values) or in a separate value buffer. The encode_constant_page function creates an encoded page with the constant layout, optional rep/def buffers, and the scalar value. At decode time, the ConstantPageScheduler loads the scalar and rep/def data, then the ConstantPageDecoder expands the scalar to fill the requested number of rows, applying nullability from the definition levels. The scheduler supports both inline values (encoded directly in the protobuf description) and buffer-backed values.

Usage

Constant encoding is selected by the compression strategy when it detects that all values in a page are identical. It is particularly effective for columns with low cardinality or default values.

Code Reference

Source Location Repository: lance-format/lance, File: rust/lance-encoding/src/encodings/logical/primitive/constant.rs, Lines: 1-515

Signature

pub(crate) fn encode_constant_page(
    column_idx: u32,
    scalar: ArrayRef,
    repdef: crate::repdef::SerializedRepDefs,
    row_number: u64,
    num_rows: u64,
) -> Result<EncodedPage>;

pub struct ConstantPageScheduler {
    buffer_offsets_and_sizes: Arc<[(u64, u64)]>,
    scalar_source: ScalarSource,
    data_type: DataType,
    def_meaning: Arc<[DefinitionInterpretation]>,
    // ...
}

impl ConstantPageScheduler {
    pub fn try_new(
        buffer_offsets_and_sizes: Arc<[(u64, u64)]>,
        inline_value: Option<Bytes>,
        data_type: DataType,
        def_meaning: Arc<[DefinitionInterpretation]>,
    ) -> Result<Self>;
}

Import use lance_encoding::encodings::logical::primitive::constant::{encode_constant_page, ConstantPageScheduler};

I/O Contract

Direction	Type	Description
Input	`ArrayRef` (single-element)	Scalar value to repeat across all rows
Input	`SerializedRepDefs`	Serialized repetition/definition levels
Output	`EncodedPage`	Page with constant layout, inline or buffered scalar, and optional rep/def
Output (decode)	Expanded `ArrayRef`	Array of repeated scalar values with nullability applied

Usage Examples

use lance_encoding::encodings::logical::primitive::constant::encode_constant_page;

let page = encode_constant_page(
    column_idx,
    scalar_array, // single-element array
    serialized_repdef,
    row_number,
    num_rows,
)?;

Related Pages

Lance_format_Lance_PrimitiveEncoding - Parent primitive encoding that selects constant pages
Lance_format_Lance_ValueEncoding - Standard value encoding (non-constant)

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment