Implementation:Run llama Llama index ObjectIndex

Overview

The ObjectIndex module defines the core abstractions for indexing and retrieving arbitrary Python objects in LlamaIndex. It provides two primary classes: ObjectRetriever which retrieves objects by converting them to and from index nodes, and ObjectIndex which wraps a standard LlamaIndex index with an object-to-node mapping layer. This module lives at llama-index-core/llama_index/core/objects/base.py (210 lines).

Purpose

The purpose of this module is to bridge the gap between arbitrary Python objects and the node-based retrieval system in LlamaIndex. Rather than requiring users to manually convert objects to nodes and back, ObjectIndex provides a generic, type-safe abstraction that handles this bidirectional mapping automatically. This is essential for use cases such as tool retrieval, SQL table selection, and any scenario where non-text objects need to be stored in and retrieved from a vector index.

Key Components

Type Variable

Name	Description
OT	A generic type variable (TypeVar("OT")) used to parameterize both ObjectRetriever and ObjectIndex with the type of objects being indexed.

Class: ObjectRetriever

A generic retriever that wraps a BaseRetriever and a BaseObjectNodeMapping to retrieve objects rather than raw nodes.

Constructor Parameters

Parameter	Type	Description
retriever	BaseRetriever	The underlying node retriever used to find relevant nodes.
object_node_mapping	BaseObjectNodeMapping[OT]	Mapping that converts between objects and nodes.
node_postprocessors	Optional[List[BaseNodePostprocessor]]	Optional list of postprocessors applied to retrieved nodes before conversion back to objects.

Properties

Property	Return Type	Description
retriever	BaseRetriever	Returns the underlying retriever instance.
object_node_mapping	BaseObjectNodeMapping[OT]	Returns the object-to-node mapping.
node_postprocessors	List[BaseNodePostprocessor]	Returns the list of node postprocessors.

Methods

Method	Parameters	Return Type	Description
retrieve	str_or_query_bundle: QueryType	List[OT]	Synchronously retrieves objects matching the query. Converts a string input to a QueryBundle, retrieves nodes, applies postprocessors, then maps nodes back to objects via from_node.
aretrieve	str_or_query_bundle: QueryType	List[OT]	Asynchronous version of retrieve. Uses await self._retriever.aretrieve() for the underlying retrieval step.

Class: ObjectIndex

A generic index wrapper that pairs a standard BaseIndex with an object-to-node mapping to provide object-level indexing and retrieval.

Constructor Parameters

Parameter	Type	Description
index	BaseIndex	The underlying LlamaIndex index for storing nodes.
object_node_mapping	BaseObjectNodeMapping	The mapping between objects and index nodes.

Properties

Property	Return Type	Description
index	BaseIndex	Returns the underlying index.
object_node_mapping	BaseObjectNodeMapping	Returns the object-to-node mapping.

Class Methods

Method	Description
from_objects(objects, object_mapping, from_node_fn, to_node_fn, index_cls, **index_kwargs)	Creates an ObjectIndex from a sequence of objects. If no object_mapping is provided, it auto-selects the best mapping using get_object_mapping(). Converts objects to nodes and builds an index (defaults to VectorStoreIndex).
from_objects_and_index(objects, index, object_mapping, from_node_fn, to_node_fn)	Creates an ObjectIndex from objects and a pre-existing index. The index is reused as-is; only the object mapping is constructed if not provided.
from_persist_dir(persist_dir, object_node_mapping)	Loads a persisted ObjectIndex from a directory. Loads the index from storage and attempts to load a SimpleObjectNodeMapping if no mapping is provided.

Instance Methods

Method	Parameters	Return Type	Description
insert_object	obj: Any	None	Adds a new object to both the object-node mapping and the underlying index.
as_retriever	node_postprocessors, **kwargs	ObjectRetriever	Returns an ObjectRetriever wrapping this index's retriever and object mapping.
as_node_retriever	**kwargs	BaseRetriever	Returns a plain node-level retriever from the underlying index (no object mapping).
persist	persist_dir, obj_node_mapping_fname	None	Persists both the object-node mapping and the index storage context. Catches NotImplementedError and PickleError for non-persistable mappings and issues a warning.

Dependencies

Module	Items Imported
pickle	Used for error handling during persistence.
warnings	Used to issue warnings when persistence fails.
llama_index.core.base.base_retriever	BaseRetriever
llama_index.core.indices.base	BaseIndex
llama_index.core.indices.vector_store.base	VectorStoreIndex (default index class)
llama_index.core.postprocessor.types	BaseNodePostprocessor
llama_index.core.objects.base_node_mapping	DEFAULT_PERSIST_FNAME, BaseObjectNodeMapping, SimpleObjectNodeMapping
llama_index.core.schema	BaseNode, QueryBundle, QueryType
llama_index.core.storage.storage_context	DEFAULT_PERSIST_DIR, StorageContext

Data Flow

The data flow for object retrieval follows this sequence:

1. User calls ObjectIndex.from_objects() with a list of Python objects.
2. The object mapping converts each object to a BaseNode via to_nodes().
3. The nodes are inserted into the underlying index (e.g., VectorStoreIndex).
4. At query time, the user calls as_retriever() to get an ObjectRetriever.
5. The ObjectRetriever.retrieve() method queries the underlying retriever for matching nodes.
6. Node postprocessors (if any) are applied to filter or rerank the results.
7. Each result node is converted back to the original object via object_node_mapping.from_node().

Persistence

The persist() method saves both the object-node mapping (via pickle) and the index storage context to disk. The from_persist_dir() class method reconstructs the ObjectIndex from a persisted directory, defaulting to SimpleObjectNodeMapping deserialization if no explicit mapping is provided. If the mapping cannot be loaded, an exception is raised.

Design Notes

• The module uses Python generics (Generic[OT]) to provide type safety for the stored objects.
• The from_objects() factory method delegates to get_object_mapping() from llama_index.core.objects.utils to auto-detect the best mapping class based on the object types.
• Both synchronous (retrieve) and asynchronous (aretrieve) retrieval paths are supported.
• The design cleanly separates the index storage layer from the object mapping layer, allowing different index backends and mapping strategies to be composed independently.