Jump to content

Connect Leeroopedia MCP: Equip your AI agents to search best practices, build plans, verify code, diagnose failures, and look up hyperparameter defaults.

Environment:Neuml Txtai Python Core Environment

From Leeroopedia


Knowledge Sources
Domains Infrastructure, NLP, Semantic_Search
Last Updated 2026-02-09 17:00 GMT

Overview

Python 3.10+ environment with PyTorch 2.4+, Transformers 4.56+, and Faiss for vector similarity search. This is the minimum environment required to run any txtai functionality.

Description

This environment defines the mandatory runtime stack for txtai v9.5.0. It requires Python 3.10 or higher and includes nine core dependencies: PyTorch for tensor computation, Hugging Face Transformers for model loading, Faiss-CPU for approximate nearest neighbor search, and supporting libraries (msgpack, huggingface-hub, numpy, pyyaml, regex, safetensors). All txtai modules — embeddings, pipelines, workflows, and agents — require this base environment. Optional functionality is enabled through extras groups (e.g., `pip install txtai[pipeline]`).

Usage

Use this environment for all txtai operations. Every txtai workflow, pipeline, embedding index, and agent requires these core dependencies. This is the mandatory prerequisite for all Implementation pages in this wiki.

System Requirements

Category Requirement Notes
OS Linux, macOS, or Windows All major platforms supported; Linux recommended for production
Python >= 3.10 Defined in `setup.py` line 158
Disk 2GB minimum For core packages; models downloaded on first use require additional space

Dependencies

System Packages

  • `python3` >= 3.10
  • `pip` (latest recommended)

Python Packages (Mandatory)

  • `faiss-cpu` >= 1.7.1.post2
  • `torch` >= 2.4
  • `transformers` >= 4.56.2
  • `msgpack` >= 1.0.7
  • `huggingface-hub` >= 0.34.0
  • `numpy` >= 1.18.4
  • `pyyaml` >= 5.3
  • `regex` >= 2022.8.17
  • `safetensors` >= 0.4.5

Optional Extras Groups

txtai provides fine-grained optional dependency groups:

  • `agent` — smolagents, jinja2, mcpadapt
  • `ann` — annoy, bitsandbytes, ggml, hnswlib, pgvector, sqlite-vec
  • `api` — fastapi, uvicorn, aiohttp
  • `cloud` — apache-libcloud
  • `console` — rich
  • `database` — duckdb, sqlalchemy, pillow
  • `graph` — networkx, grand-cypher, grand-graph
  • `model` — onnx, onnxruntime
  • `pipeline` — audio, data, image, llm, text, train subgroups
  • `scoring` — sqlalchemy
  • `vectors` — sentence-transformers, litellm, llama-cpp-python, model2vec
  • `workflow` — croniter, pandas, requests
  • `all` — installs everything except dev dependencies

Credentials

No credentials are required for the core environment. Optional modules may require:

  • `HF_TOKEN`: Hugging Face Hub token for gated model access (optional)
  • `TOKENIZERS_PARALLELISM`: Set to `false` to suppress tokenizer parallelism warnings

Quick Install

# Minimal install (core only)
pip install txtai

# Full install (all optional dependencies)
pip install txtai[all]

# Specific extras
pip install txtai[pipeline,api,vectors]

# CPU-only PyTorch (reduces install size)
pip install torch==2.10.0+cpu -f https://download.pytorch.org/whl/torch
pip install txtai

Code Evidence

Core dependency definitions from `setup.py:8-12`:

# Required dependencies
install = ["faiss-cpu>=1.7.1.post2", "msgpack>=1.0.7", "torch>=2.4", "transformers>=4.56.2"]

# Required dependencies that are also base transformers dependencies
install += ["huggingface-hub>=0.34.0", "numpy>=1.18.4", "pyyaml>=5.3", "regex>=2022.8.17", "safetensors>=0.4.5"]

Python version constraint from `setup.py:158`:

python_requires=">=3.10",

Optional dependency pattern from `src/python/txtai/agent/__init__.py:5-12`:

try:
    from .base import Agent
except ImportError:
    from .placeholder import Agent
# Raises: "Agent is not available - install 'agent' extra to enable"

Common Errors

Error Message Cause Solution
`ModuleNotFoundError: No module named 'txtai'` txtai not installed `pip install txtai`
`Agent is not available - install 'agent' extra to enable` smolagents not installed `pip install txtai[agent]`
`API is not available - install 'api' extra to enable` fastapi not installed `pip install txtai[api]`
`Console is not available - install 'console' extra to enable` rich not installed `pip install txtai[console]`
`zsh: no matches found: txtai[pipeline]` Shell interprets brackets `pip install 'txtai[pipeline]'` (quote the argument)

Compatibility Notes

  • macOS (zsh): Escape square brackets when installing extras: `pip install 'txtai[pipeline]'`
  • Windows (PowerShell): Same bracket escaping applies
  • ARM64 (Apple Silicon, AWS Graviton): CPU-only PyTorch is installed by default in Docker builds
  • Older Faiss versions: May require `brew install libomp` on macOS

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment