Implementation:Lance format Lance NamespaceSchema

Knowledge Sources	Lance
Domains	Namespace, Schema
Last Updated	2026-02-08 19:33 GMT

Overview

Description

The NamespaceSchema module provides conversion utilities between Apache Arrow schema types and the JsonArrow* schema representations used in the Lance Namespace REST protocol. This enables Arrow schemas to be serialized to JSON for transmission over REST APIs and deserialized back into Arrow types on the receiving end.

The module provides three primary conversion functions:

• arrow_schema_to_json -- Converts an arrow::datatypes::Schema to a JsonArrowSchema
• arrow_field_to_json (private) -- Converts individual Field objects to JsonArrowField
• arrow_type_to_json (private) -- Converts DataType variants to JsonArrowDataType

The module handles the full range of Arrow data types including:

• Primitive types (null, bool, integers, floats, decimals, dates, times, timestamps, durations, intervals)
• String and binary types (utf8, large_utf8, binary, large_binary, fixed_size_binary)
• Nested types (list, large_list, fixed_size_list, struct, map)

For decimal types, precision and scale are encoded into a single length field using the formula precision * 1000 + scale.

Usage

This module is used internally by namespace implementations when they need to communicate Arrow schema information over the REST API. The JsonArrowSchema format is the wire representation defined in the Lance Namespace specification.

Code Reference

Source Location

rust/lance-namespace/src/schema.rs

Signature

pub fn arrow_schema_to_json(arrow_schema: &ArrowSchema) -> Result<JsonArrowSchema>;

Import

use lance_namespace::schema::arrow_schema_to_json;

I/O Contract

Inputs

Parameter	Type	Description
arrow_schema	&ArrowSchema	An Apache Arrow schema to convert to JSON representation

Outputs

Type	Description
Result<JsonArrowSchema>	A JSON-serializable schema representation containing fields and optional metadata

JsonArrowSchema Structure

Field	Type	Description
fields	Vec<JsonArrowField>	List of field definitions with name, nullable flag, type, and optional metadata
metadata	Option<HashMap<String, String>>	Optional schema-level metadata (omitted when empty)

Usage Examples

use arrow::datatypes::{DataType, Field, Schema as ArrowSchema};
use lance_namespace::schema::arrow_schema_to_json;

let schema = ArrowSchema::new(vec![
    Field::new("id", DataType::Int64, false),
    Field::new("name", DataType::Utf8, true),
    Field::new("embedding", DataType::FixedSizeList(
        Field::new("item", DataType::Float32, false).into(), 128
    ), false),
]);

let json_schema = arrow_schema_to_json(&schema).unwrap();
// json_schema can now be serialized and sent over REST

Related Pages

• Lance_format_Lance_LanceNamespaceTrait -- Namespace trait whose table operations use this schema conversion
• Lance_format_Lance_NamespaceError -- Error types used when conversion fails
• Lance_format_Lance_RestApi -- REST API specification that consumes JsonArrowSchema