Implementation:Lance format Lance Java JsonFields

Knowledge Sources	Lance
Domains	Java_SDK, Dataset_Management
Last Updated	2026-02-08 19:33 GMT

Overview

Description

The JsonFields utility class provides static factory methods for constructing Arrow Field objects annotated with the arrow.json extension type. This extension type marks Utf8 or LargeUtf8 fields as logically carrying JSON text.

When writing data, Lance converts fields annotated with arrow.json into its internal JSONB representation (physically stored as LargeBinary with the lance.json extension name). When reading, Lance converts lance.json back to arrow.json (Utf8), so callers always work with JSON text rather than binary JSON. The internal lance.json storage type is intentionally not exposed to keep the JSONB format an implementation detail.

The class is final with a private constructor, following the utility class pattern. It uses the ARROW:extension:name metadata key with the value arrow.json to annotate fields.

Usage

Use JsonFields.jsonUtf8() or JsonFields.jsonLargeUtf8() when defining schema fields that will store JSON data. This ensures Lance handles the JSON-to-JSONB conversion automatically on write and the reverse conversion on read, providing efficient binary storage with text-based API access.

Code Reference

Source Location

java/src/main/java/org/lance/util/JsonFields.java

Signature

public final class JsonFields {
    public static Field jsonUtf8(String name, boolean nullable);
    public static Field jsonLargeUtf8(String name, boolean nullable);
}

Import

import org.lance.util.JsonFields;

I/O Contract

Inputs

Parameter	Type	Required	Description
name	String	Yes	The field name
nullable	boolean	Yes	Whether the field allows null values

Outputs

Method	Return Type	Description
jsonUtf8()	Field	Arrow Field with Utf8 storage type and arrow.json extension metadata
jsonLargeUtf8()	Field	Arrow Field with LargeUtf8 storage type and arrow.json extension metadata

Extension Metadata Applied

Key	Value
ARROW:extension:name	arrow.json

Usage Examples

// Create a nullable JSON field using Utf8 storage
Field jsonField = JsonFields.jsonUtf8("metadata", true);

// Create a non-nullable JSON field using LargeUtf8 storage
Field largeJsonField = JsonFields.jsonLargeUtf8("document", false);

// Use in a schema definition
Schema schema = new Schema(List.of(
    Field.nullable("id", new ArrowType.Int(64, true)),
    Field.nullable("name", new ArrowType.Utf8()),
    JsonFields.jsonUtf8("properties", true),
    JsonFields.jsonLargeUtf8("raw_data", true)
));

// Lance automatically handles JSON <-> JSONB conversion:
// - On write: arrow.json (Utf8 text) -> lance.json (LargeBinary JSONB)
// - On read:  lance.json (LargeBinary JSONB) -> arrow.json (Utf8 text)

Related Pages

• Lance_format_Lance_Java_LanceField -- Lance field representation with logical type and Arrow type mapping
• Lance_format_Lance_Java_LanceSchema -- Lance schema containing fields and metadata
• Lance_format_Lance_Java_ColumnAlteration -- Alters column types, which may interact with JSON fields