Implementation:Lance format Lance StructEncoding

Knowledge Sources	Lance
Domains	Encoding, Compression
Last Updated	2026-02-08 19:33 GMT

Overview

StructuralStructScheduler and related types handle the encoding and decoding of struct (record) fields by coordinating the encoding of each child field and merging their results back into struct arrays.

Description

Struct encoding decomposes a struct array into its child fields, encoding each independently while tracking struct-level validity in the rep/def builder. The key components include:

• StructStructuralEncoder: Encodes struct arrays by distributing child arrays to their respective child encoders. Adds struct validity to the rep/def builder and passes each child's data to the appropriate encoder.
• StructuralStructScheduler: Schedules reads for struct fields by coordinating child schedulers. Implements a min-heap strategy that schedules the child with the least rows first, enabling complete rows to be assembled as quickly as possible.
• RepDefStructSchedulingJob: Uses a BinaryHeap to prioritize scheduling children with the fewest scheduled rows, ensuring balanced I/O across children.
• StructuralStructDecoder: Decodes struct arrays by collecting decoded child arrays and reassembling them into a StructArray with proper validity.

Usage

Use this encoding for struct-typed columns and for the top-level record batch structure. It is automatically selected when the schema contains struct fields.

Code Reference

Source Location	Repository: lance-format/lance, File: rust/lance-encoding/src/encodings/logical/struct.rs, Lines: 1-810
Signature	pub struct StructuralStructScheduler { children: Vec<(Box<dyn StructuralFieldScheduler>, Arc<arrow_schema::Field>)>, child_fields: Vec<Arc<arrow_schema::Field>>, } pub struct StructStructuralEncoder { children: Vec<Box<dyn FieldEncoder>>, } pub struct StructuralStructDecoder { children: Vec<Box<dyn StructuralFieldDecoder>>, child_fields: Fields, }
Import	use lance_encoding::encodings::logical::r#struct::{StructuralStructScheduler, StructStructuralEncoder};

I/O Contract

Direction	Type	Description
Input	ArrayRef (Struct)	Struct array with child arrays
Input	RepDefBuilder	Builder for struct-level validity
Output	Vec<EncodeTask>	Merged encode tasks from all child encoders
Output (decode)	DecodedArray	Reconstructed StructArray with all children and validity

Usage Examples

use lance_encoding::encodings::logical::r#struct::StructStructuralEncoder;

// Created by the encoding framework for Struct<...> columns
let encoder = StructStructuralEncoder {
    children: vec![child_encoder_a, child_encoder_b],
};

Related Pages

• Lance_format_Lance_PrimitiveEncoding - Leaf-level encoding for struct children
• Lance_format_Lance_ListEncoding - List encoding (may be a struct child)
• Lance_format_Lance_MapEncoding - Map encoding (delegates entries to struct)
• Lance_format_Lance_PackedEncoding - Packed struct for fixed-width struct optimization