Jump to content

Connect SuperML | Leeroopedia MCP: Equip your AI agents with best practices, code verification, and debugging knowledge. Powered by Leeroo — building Organizational Superintelligence. Contact us at founders@leeroo.com.

Implementation:Run llama Llama index StorageContext Persist

From Leeroopedia
Revision as of 11:49, 16 February 2026 by Admin (talk | contribs) (Auto-imported from implementations/Run_llama_Llama_index_StorageContext_Persist.md)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Knowledge Sources
Domains RAG, Data_Management
Last Updated 2026-02-11 00:00 GMT

Overview

Concrete dataclass and associated utility function for persisting and reloading index state across all storage backends managed by LlamaIndex.

Description

The StorageContext is a dataclass that aggregates all storage backends (document store, index store, vector stores, graph store, property graph store) into a single coordinated unit. Its persist method serializes each store to a designated file within a target directory. The companion function load_index_from_storage reconstructs an index from a previously persisted StorageContext, bypassing the need to recompute embeddings.

Usage

Use StorageContext.from_defaults() to create a new context (optionally with custom backends) or to reload from a persist directory. Call persist() after building an index to save state. Use load_index_from_storage() at application startup to reload a pre-built index.

Code Reference

Source Location

  • Repository: run-llama/llama_index
  • File: llama-index-core/llama_index/core/storage/storage_context.py
  • Lines: L52-280 (StorageContext class)

Signature

@dataclass
class StorageContext:
    # --- Fields ---
    docstore: BaseDocumentStore
    index_store: BaseIndexStore
    vector_stores: Dict[str, BasePydanticVectorStore]
    graph_store: GraphStore
    property_graph_store: Optional[PropertyGraphStore]

    @classmethod
    def from_defaults(
        cls,
        docstore: Optional[BaseDocumentStore] = None,
        index_store: Optional[BaseIndexStore] = None,
        vector_store: Optional[BasePydanticVectorStore] = None,
        image_store: Optional[BasePydanticVectorStore] = None,
        vector_stores: Optional[Dict[str, BasePydanticVectorStore]] = None,
        graph_store: Optional[GraphStore] = None,
        property_graph_store: Optional[PropertyGraphStore] = None,
        persist_dir: Optional[str] = None,
        fs: Optional[fsspec.AbstractFileSystem] = None,
    ) -> "StorageContext":
        ...

    def persist(
        self,
        persist_dir: str = "./storage",
        docstore_fname: str = DOCSTORE_FNAME,
        index_store_fname: str = INDEX_STORE_FNAME,
        vector_store_fname: str = VECTOR_STORE_FNAME,
        image_store_fname: str = IMAGE_STORE_FNAME,
        graph_store_fname: str = GRAPH_STORE_FNAME,
        pg_graph_store_fname: str = PG_GRAPH_STORE_FNAME,
        fs: Optional[fsspec.AbstractFileSystem] = None,
    ) -> None:
        ...
# load_index_from_storage utility function
def load_index_from_storage(
    storage_context: StorageContext,
    index_id: Optional[str] = None,
    **kwargs,
) -> BaseIndex:
    ...

Import

from llama_index.core import StorageContext, load_index_from_storage

I/O Contract

Inputs (from_defaults)

Name Type Required Description
docstore BaseDocumentStore or None No Custom document store; defaults to SimpleDocumentStore
index_store BaseIndexStore or None No Custom index store; defaults to SimpleIndexStore
vector_store BasePydanticVectorStore or None No Custom vector store for the default namespace
image_store BasePydanticVectorStore or None No Custom vector store for image embeddings
vector_stores Dict[str, BasePydanticVectorStore] or None No Multiple named vector stores
graph_store GraphStore or None No Custom graph store; defaults to SimpleGraphStore
property_graph_store PropertyGraphStore or None No Custom property graph store
persist_dir str or None No Directory to load previously persisted state from
fs fsspec.AbstractFileSystem or None No Filesystem abstraction (e.g., for S3 or GCS)

Inputs (persist)

Name Type Required Description
persist_dir str No (default: "./storage") Target directory for serialized store files
docstore_fname str No Filename for the document store JSON
index_store_fname str No Filename for the index store JSON
vector_store_fname str No Filename for the vector store JSON
image_store_fname str No Filename for the image store JSON
graph_store_fname str No Filename for the graph store JSON
pg_graph_store_fname str No Filename for the property graph store JSON
fs fsspec.AbstractFileSystem or None No Filesystem abstraction for remote storage

Inputs (load_index_from_storage)

Name Type Required Description
storage_context StorageContext Yes StorageContext loaded from a persist directory
index_id str or None No Specific index ID to load; if None, loads the single index present

Outputs

Method Return Type Description
from_defaults StorageContext Initialized storage context with all stores configured
persist None Writes store files to disk (side effect)
load_index_from_storage BaseIndex Fully reconstructed index ready for querying

Usage Examples

Persist Index to Disk

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

# Build index (computes embeddings)
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# Persist all stores to disk
index.storage_context.persist(persist_dir="./storage")

Reload Index from Disk

from llama_index.core import StorageContext, load_index_from_storage

# Reload without recomputing embeddings
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)

# Ready to query immediately
query_engine = index.as_query_engine()
response = query_engine.query("What is the summary?")

Custom Storage Backends

from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.vector_stores.chroma import ChromaVectorStore
import chromadb

# Use Chroma as the vector store backend
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("my_docs")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Build storage context with custom vector store
storage_context = StorageContext.from_defaults(
    vector_store=vector_store,
)

# Build index using the custom storage context
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context,
)

# Persist non-vector stores (Chroma handles its own persistence)
index.storage_context.persist(persist_dir="./storage")

Reload with External Vector Store

from llama_index.core import StorageContext, load_index_from_storage
from llama_index.vector_stores.chroma import ChromaVectorStore
import chromadb

# Reconnect to the external vector store
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_collection("my_docs")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Load storage context with external vector store + local metadata
storage_context = StorageContext.from_defaults(
    vector_store=vector_store,
    persist_dir="./storage",
)
index = load_index_from_storage(storage_context)

Related Pages

Implements Principle

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment