Environment:Deepset ai Haystack HuggingFace Model Environment

Knowledge Sources	Haystack Sentence Transformers HuggingFace Transformers
Domains	Infrastructure, NLP, Deep_Learning
Last Updated	2026-02-11 20:00 GMT

Overview

HuggingFace model environment with Sentence Transformers >= 5.0.0, Transformers >= 4.57, and optional HuggingFace API token for embedding, ranking, and extractive reading components.

Description

This environment provides the ML model infrastructure needed for Haystack components that run local transformer models. It includes Sentence Transformers for embedding generation, HuggingFace Transformers for cross-encoder ranking and extractive QA, and PyTorch as the deep learning backend. These are optional dependencies not included in the base `haystack-ai` install and must be installed separately. Components use the `LazyImport` pattern to defer import errors until runtime.

Usage

Use this environment when running any pipeline that includes SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, TransformersSimilarityRanker, ExtractiveReader, SASEvaluator, or DocumentLanguageClassifier. These components load and run local transformer models, requiring PyTorch and the transformers ecosystem.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, Windows	All platforms supported; Linux recommended for GPU
Hardware	CPU minimum; GPU recommended	GPU significantly speeds embedding/inference
RAM	4GB minimum	Model-dependent; large models need more
Disk	1-10GB	For downloaded model weights (cached in ~/.cache/huggingface)

Dependencies

Python Packages

`sentence-transformers` >= 5.0.0
`transformers[torch, sentencepiece]` >= 4.57, < 5
`huggingface_hub` >= 0.27.0
`torch` (installed via transformers[torch])

Optional Packages

`langdetect` (for DocumentLanguageClassifier)
`nltk` (for sentence-aware splitting in DocumentSplitter)

Credentials

The following environment variables are optional but recommended for accessing private or gated models:

`HF_API_TOKEN`: HuggingFace API token (primary)
`HF_TOKEN`: Alternative HuggingFace token variable

Note: Components check both variables in order via `Secret.from_env_var(["HF_API_TOKEN", "HF_TOKEN"], strict=False)`. The token is optional (strict=False) for public models.

Quick Install

# Install Haystack with HuggingFace model support
pip install haystack-ai "sentence-transformers>=5.0.0" "transformers[torch,sentencepiece]>=4.57,<5"

# Optional: for language detection
pip install langdetect

# Optional: set HuggingFace token for private models
export HF_API_TOKEN="hf_..."

Code Evidence

Lazy import pattern from `haystack/utils/hf.py:25-30`:

with LazyImport(message="Run 'pip install transformers[torch]'") as torch_import:
    import torch

with LazyImport(message="Run 'pip install transformers'") as transformers_import:
    from transformers import PreTrainedTokenizerBase

HuggingFace token resolution from `haystack/components/embedders/sentence_transformers_text_embedder.py:42`:

token: Secret | None = Secret.from_env_var(["HF_API_TOKEN", "HF_TOKEN"], strict=False),

Test dependency versions from `pyproject.toml:88-96`:

"transformers[torch, sentencepiece]>=4.57,<5",
"huggingface_hub>=0.27.0",
"sentence-transformers>=5.0.0",

Common Errors

Error Message	Cause	Solution
`LazyImportError: Run 'pip install transformers[torch]'`	Transformers/PyTorch not installed	`pip install "transformers[torch]>=4.57,<5"`
`LazyImportError: Run 'pip install sentence-transformers'`	Sentence Transformers not installed	`pip install "sentence-transformers>=5.0.0"`
`OSError: model not found`	Model not on disk and no internet	Set `local_files_only=False` or download model manually
`401 Client Error: Unauthorized`	Accessing gated model without token	Set `HF_API_TOKEN` or `HF_TOKEN` environment variable

Compatibility Notes

Sentence Transformers >= 5.0.0: Major version jump; older versions may have incompatible APIs
Transformers < 5: Upper bound ensures API stability; Transformers 5.x may break compatibility
Backend options: Embedders support `torch`, `onnx`, and `openvino` backends for inference
Model caching: Models are cached in `~/.cache/huggingface/hub` by default

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment