Implementation:Neuml Txtai LiteLLM Vectors

Knowledge Sources	Neuml_Txtai
Domains	Embeddings, Vectors, API Integration, LLM
Last Updated	2026-02-10 01:00 GMT

Overview

Concrete tool for building embedding vectors via external API calls through LiteLLM provided by txtai.

Description

The LiteLLM class extends the base Vectors class to generate embeddings using the LiteLLM library, which provides a unified interface to over 100+ embedding API providers (OpenAI, Cohere, Azure, Bedrock, etc.).

Key features:

Model detection: The static ismodel method checks whether a given path corresponds to a LiteLLM-supported provider by calling litellm.get_llm_provider(path). It excludes models available on the Hugging Face Hub (checked via cached_file) to avoid conflicts with local model backends.
Server-side batching: The encode method delegates batching entirely to the remote API by passing the full input list to litellm.embedding().
No local model: The loadmodel method returns None since the model runs remotely.
Additional parameters: Extra API parameters can be passed via the vectors config key, which are forwarded as keyword arguments to the LiteLLM embedding call.

The response is parsed by extracting the embedding field from each element in response.data and constructing a float32 NumPy array.

Usage

Use LiteLLM vectors when you want to generate embeddings through a remote API service rather than running a model locally. This is ideal for using commercial embedding APIs (OpenAI, Cohere, etc.), serverless deployments where GPU resources are not available, or when you need to switch between embedding providers without code changes.

Code Reference

Source Location

Repository: Neuml_Txtai
File: src/python/txtai/vectors/dense/litellm.py

Signature

class LiteLLM(Vectors):
    @staticmethod
    def ismodel(path) -> bool
    @staticmethod
    def ishub(path) -> bool
    def __init__(self, config, scoring, models)
    def loadmodel(self, path) -> None
    def encode(self, data, category=None) -> ndarray

Import

from txtai.vectors.dense.litellm import LiteLLM

I/O Contract

Inputs

Name	Type	Required	Description
config	dict	Yes	Configuration dictionary. Must include path (str, the LiteLLM model identifier, e.g., "openai/text-embedding-3-small"). Optionally includes vectors (dict of additional keyword arguments passed to litellm.embedding()).
scoring	Scoring	No	Optional scoring instance for token weighting.
models	object	No	Shared models cache instance.
data	list[str]	Yes (encode)	List of text strings to encode.
category	str	No	Optional category hint (not used).
path (ismodel)	str	Yes (ismodel)	Model path to check for LiteLLM provider compatibility.

Outputs

Name	Type	Description
embeddings	ndarray (float32)	NumPy array of embedding vectors with shape (n, dimensions).
ismodel	bool	True if the path is recognized as a LiteLLM-supported provider and is not a HF Hub model.
ishub	bool	True if the path points to a model on the Hugging Face Hub.

Usage Examples

from txtai.embeddings import Embeddings

# Use OpenAI embeddings via LiteLLM
embeddings = Embeddings({
    "path": "openai/text-embedding-3-small",
    "dimensions": 1536
})

# Index documents
embeddings.index([
    (0, "machine learning fundamentals", None),
    (1, "natural language processing with transformers", None),
    (2, "computer vision and image recognition", None),
])

# Search
results = embeddings.search("deep learning NLP", limit=5)

# Use Cohere embeddings with additional parameters
embeddings = Embeddings({
    "path": "cohere/embed-english-v3.0",
    "vectors": {"input_type": "search_document"},
    "dimensions": 1024
})

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment