Environment:Confident ai Deepeval Python 3 9 Runtime

Knowledge Sources	deepeval PyPI deepeval
Domains	Infrastructure, LLM_Evaluation
Last Updated	2026-02-14 10:00 GMT

Overview

Python 3.9+ runtime environment with core dependencies for the deepeval LLM evaluation framework (v3.8.4).

Description

This environment provides the base runtime for running deepeval, an LLM evaluation framework. It requires Python 3.9 or higher (but below 4.0) and includes a substantial set of core dependencies for HTTP communication, async execution, CLI tooling, OpenTelemetry tracing, and Pydantic-based configuration. The framework is CPU-based by default; GPU is only needed for optional NLP scoring features (BERTScore).

Usage

Use this environment for all deepeval operations including metric evaluation, test case construction, dataset management, CLI commands, and tracing. This is the mandatory base environment for every deepeval workflow.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, Windows	Linux recommended for production; Windows needs WSL2 for some features
Python	>= 3.9, < 4.0	Defined in `pyproject.toml:19`
Disk	500MB+	For package installation and `.deepeval` cache directory

Dependencies

System Packages

No OS-level system packages required for core functionality

Python Packages (Core)

`openai` (any version) - Default LLM provider SDK
`pydantic` >= 2.11.7 - Data validation and settings management
`pydantic-settings` >= 2.10.1 - Environment-based configuration
`python-dotenv` >= 1.1.1 - Dotenv file loading
`requests` >= 2.31.0 - HTTP client
`rich` >= 13.6.0, < 15.0.0 - Terminal output formatting
`typer` >= 0.9, < 1.0.0 - CLI framework
`click` >= 8.0.0, < 8.4.0 - CLI dependency
`tenacity` >= 8.0.0, <= 10.0.0 - Retry logic
`opentelemetry-api` >= 1.24.0 - OpenTelemetry tracing API
`opentelemetry-sdk` >= 1.24.0 - OpenTelemetry tracing SDK
`grpcio` >= 1.67.1 - gRPC for telemetry export
`posthog` >= 5.4.0, < 7.0.0 - Anonymous usage telemetry
`nest_asyncio` (any version) - Nested event loop support
`aiohttp` (any version) - Async HTTP
`pytest` (any version) - Test framework integration
`pytest-xdist` (any version) - Parallel test execution
`pytest-asyncio` (any version) - Async test support
`tabulate` >= 0.9.0 - Table formatting
`portalocker` (any version) - File locking for test run cache
`tqdm` >= 4.66.1 - Progress bars
`sentry-sdk` (any version) - Error reporting
`pyfiglet` (any version) - ASCII art for CLI
`jinja2` (any version) - Template rendering
`setuptools` (any version) - Package utilities
`wheel` (any version) - Package distribution

Python Packages (Optional)

`pandas` - Required for CSV dataset import/export
`langchain`, `langchain_core`, `langchain_community`, `langchain_text_splitters` - Required for document chunking in synthesizer
`chromadb` - Required for context generation in synthesizer
`tiktoken` - Required for token counting in document chunking
`pypdf` - Required for PDF document loading
`docx2txt` - Required for DOCX document loading
`bert-score` - Required for BERTScore NLP metric
`torch` - Required for BERTScore (GPU optional)
`sentence_transformers` - Required for HallucinationMetric
`anthropic` - Required for Anthropic tracing integration
`crewai` - Required for CrewAI integration (Python >= 3.10)
`pydantic-ai` - Required for PydanticAI integration (Python >= 3.10)
`llama-index` >= 0.14.4 - Required for LlamaIndex integration
`openai-agents` >= 0.3.3 - Required for OpenAI Agents integration

Credentials

The following environment variables may be needed depending on which LLM provider and features are used:

Confident AI Platform:

`CONFIDENT_API_KEY` or `API_KEY`: Confident AI API key for uploading evaluation results and tracing
`CONFIDENT_BASE_URL`: Custom Confident AI API endpoint (optional)

LLM Provider Keys (set one based on your chosen provider):

`OPENAI_API_KEY`: OpenAI API key (default provider)
`ANTHROPIC_API_KEY`: Anthropic API key
`AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`: AWS credentials for Bedrock
`AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT`: Azure OpenAI credentials
`GOOGLE_API_KEY`: Google/Gemini API key
`DEEPSEEK_API_KEY`: DeepSeek API key
`GROK_API_KEY`: Grok API key
`LITELLM_API_KEY`: LiteLLM API key
`MOONSHOT_API_KEY`: Kimi/Moonshot API key
`OPENROUTER_API_KEY`: OpenRouter API key
`PORTKEY_API_KEY`: Portkey API key

Quick Install

# Install core deepeval
pip install deepeval

# With optional synthesizer dependencies
pip install deepeval langchain langchain-core langchain-community langchain-text-splitters chromadb tiktoken

# With optional NLP scoring
pip install deepeval bert-score torch

# With specific integration
pip install deepeval anthropic       # Anthropic tracing
pip install deepeval crewai          # CrewAI integration (Python 3.10+)
pip install deepeval pydantic-ai     # PydanticAI integration (Python 3.10+)
pip install deepeval llama-index     # LlamaIndex integration
pip install deepeval openai-agents   # OpenAI Agents integration

Code Evidence

Python version constraint from `pyproject.toml:19`:

[tool.poetry.dependencies]
python = ">=3.9, <4.0"

Pydantic v2 detection from `deepeval/utils.py:39`:

PYDANTIC_V2 = pydantic.VERSION.startswith("2")

Pydantic v1/v2 compatibility layer from `deepeval/utils.py:42-72`:

def make_model_config(**kwargs):
    if PYDANTIC_V2:
        from pydantic import ConfigDict
        return ConfigDict(**kwargs)
    else:
        class Config:
            pass
        for key, value in kwargs.items():
            setattr(Config, key, value)
        return Config

Optional dependency guards from `deepeval/synthesizer/chunking/doc_chunker.py:23-53`:

def _get_langchain():
    global _langchain_ns, _langchain_import_error
    if _langchain_ns is not None:
        return _langchain_ns
    try:
        from langchain_core.documents import Document as LCDocument
        from langchain_text_splitters import TokenTextSplitter
        # ...
    except Exception as e:
        raise ImportError(
            f"langchain, langchain_community, and langchain_text_splitters are required. Root cause: {e}"
        )

Optional pandas guard from `deepeval/dataset/dataset.py:285-289`:

try:
    import pandas as pd
except ModuleNotFoundError:
    raise ModuleNotFoundError(
        "Please install pandas to use this method. 'pip install pandas'"
    )

Optional BERTScore and torch from `deepeval/scorer/scorer.py:157-170`:

try:
    from bert_score import BERTScorer
except ModuleNotFoundError as e:
    print("Please install bert_score module. Command: pip install bert-score")
try:
    import torch
except ModuleNotFoundError as e:
    print("Please install torch module. Command: pip install torch")
# FIXME: Fix the case for mps
device = "cuda" if torch.cuda.is_available() else "cpu"

Integration Python version constraints from `pyproject.toml:91-94`:

crewai = { version = "*", python = ">=3.10,<3.14" }
pydantic-ai = { version = "*", python = ">=3.10,<3.14" }
llama-index = "^0.14.4"
openai-agents = "^0.3.3"

Common Errors

Error Message	Cause	Solution
`ModuleNotFoundError: Please install pandas to use this method`	pandas not installed when using CSV export/import	`pip install pandas`
`ImportError: langchain, langchain_community, and langchain_text_splitters are required`	Missing synthesizer dependencies	`pip install langchain langchain-core langchain-community langchain-text-splitters`
`ImportError: chromadb is required for this functionality`	chromadb not installed for context generation	`pip install chromadb`
`ModuleNotFoundError: Please install bert_score module`	BERTScore not installed for NLP scoring	`pip install bert-score`
`ModuleNotFoundError: Please install torch module`	PyTorch not installed for GPU-based scoring	`pip install torch`
`ImportError: The 'sentence_transformers' library is required to use the HallucinationMetric`	sentence_transformers not installed	`pip install sentence-transformers`
`ModuleNotFoundError: Please install OpenAI to use this feature`	OpenAI SDK missing for tracing integration	`pip install openai`
`ModuleNotFoundError: Please install anthropic to use this feature`	Anthropic SDK missing for tracing integration	`pip install anthropic`

Compatibility Notes

Python 3.10+ Required: CrewAI and PydanticAI integrations require Python >= 3.10 (< 3.14). The core framework works with Python 3.9+.
Pydantic v2 Required: While the codebase has v1 compatibility shims (`PYDANTIC_V2` check in `utils.py`), `pyproject.toml` pins `pydantic >= 2.11.7`, so v2 is the only supported version.
GPU Optional: CUDA GPU is only needed for `BERTScorer` in `deepeval/scorer/scorer.py`. The MPS (Apple Silicon) path has a known FIXME (`# FIXME: Fix the case for mps`). All other features are CPU-only.
Windows: The `ContextGenerator` includes a Windows-specific workaround in `safe_rmtree` that uses `attrib -r -s -h` to handle locked ChromaDB files (`deepeval/synthesizer/chunking/context_generator.py:51`).
Read-Only Filesystems: The test run cache gracefully falls back when running on read-only filesystems, logging a warning instead of failing (`deepeval/test_run/cache.py:32`).

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment