Jump to content

Connect SuperML | Leeroopedia MCP: Equip your AI agents with best practices, code verification, and debugging knowledge. Powered by Leeroo — building Organizational Superintelligence. Contact us at founders@leeroo.com.

Heuristic:Lance format Lance Warning Deprecated Legacy Encodings

From Leeroopedia
Revision as of 10:54, 16 February 2026 by Admin (talk | contribs) (Auto-imported from heuristics/Lance_format_Lance_Warning_Deprecated_Legacy_Encodings.md)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)



Knowledge Sources
Domains Encoding, Deprecation
Last Updated 2026-02-08 22:00 GMT

Overview

DEPRECATION WARNING: The legacy encoding and file implementations in rust/lance-encoding/src/previous/ and rust/lance-file/src/previous/ are retained solely for backward compatibility with Lance v1/v2.0 format files. They are not used for new data written in Lance v2.1+.

Description

The Lance project maintains two sets of legacy code in previous/ directories:

  • lance-encoding/src/previous/ -- Legacy 2.0 format encoders and decoders. The module comment states: "Legacy code for the 2.0 format that is no longer used in 2.1+." This includes both logical encodings (primitive, struct, blob, binary, list) and physical encodings (packed_struct, fixed_size_list, bitpack, bitmap, value, basic, binary, block, fixed_size_binary, dictionary, fsst).
  • lance-file/src/previous/ -- Legacy Lance file v1 reader, writer, page table, and format definitions. The module comment states: "Legacy Lance file v1 implementation kept for backwards compatibility."

These are not candidates for removal because they are needed to read datasets written with older library versions. However, they should not be used as reference for new development.

Usage

Reference this warning when:

  • Working on encoding or file format code -- do not extend or modify legacy code paths
  • Encountering performance issues with older datasets -- consider migrating data to the current format
  • Planning new encoding features -- build on the current 2.1+ encoding pipeline, not the legacy one

The Insight (Rule of Thumb)

  • Action: Do not modify or extend legacy encoding/file code. All new encoding work should target the current encoding pipeline in rust/lance-encoding/src/ (not the previous/ subdirectory).
  • Value: 18 legacy encoding/file modules are maintained read-only for backward compatibility.
  • Trade-off: Code size and maintenance burden. These modules add complexity but are essential for reading older datasets.
  • Migration: Users can compact/rewrite old datasets to upgrade them to the current format using compact_files().

Reasoning

Lance follows a copy-on-write versioning model. Old data files persist until compacted. The legacy readers must remain functional to support reading these older format files. The encoding format has evolved significantly from v1 to v2.0 to v2.1, with each generation improving compression ratios and read performance. The FLAG_USE_V2_FORMAT_DEPRECATED feature flag in lance-table/src/feature_flags.rs is also deprecated as the v2 format flag is no longer used in current code.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment
Retrieved from "https://leeroopedia.com/index.php?title=Heuristic:Lance_format_Lance_Warning_Deprecated_Legacy_Encodings&oldid=960"