Environment:Explodinggradients Ragas Optional Metrics Environment

Knowledge Sources	Ragas pyproject.toml
Domains	NLP, LLM_Evaluation, Metrics
Last Updated	2026-02-10 12:00 GMT

Overview

Optional Python packages required by specialized Ragas metrics including ROUGE, BLEU, chrF, string similarity, data comparison, HuggingFace NLI, and statistical correlation.

Description

Ragas core metrics (Faithfulness, AspectCritic, ContextPrecision, etc.) only require the base dependencies and an LLM provider. However, several specialized metrics depend on additional packages that are not installed by default. Each package is guarded by a `try/except ImportError` block and raises a clear error message when the dependency is missing. This environment documents all optional metric dependencies.

Usage

Install these packages when you need to use specific non-LLM metrics or statistical correlation analysis. Most LLM-as-Judge metrics do not need any of these packages. The `ragas[all]` extra installs the most common ones.

System Requirements

Category	Requirement	Notes
Python	>= 3.9	Base Ragas requirement
Hardware	CPU-only for most; GPU optional for HuggingFace HHEM	`FaithfulnesswithHHEM` defaults to CPU but supports GPU via `device` parameter

Dependencies

Text Similarity Metrics

`rouge_score` -- Required by: `RougeScore` metric (`src/ragas/metrics/_rouge_score.py:21-27`, `src/ragas/metrics/collections/_rouge_score.py:74-75`)
`sacrebleu` -- Required by: `BleuScore` metric (`src/ragas/metrics/_bleu_score.py:21-26`), `ChrfScore` metric (`src/ragas/metrics/_chrf_score.py:21-26`), and their collection counterparts
`rapidfuzz` -- Required by: `NonLLMStringSimilarity` metric with Levenshtein, Hamming, Jaro, JaroWinkler distances (`src/ragas/metrics/_string.py:70-75`); also used in testset relationship builders (`src/ragas/testset/transforms/relationship_builders/traditional.py:90-96`)

Data Comparison

`pandas` -- Required by: `DataCompyScore` metric, `to_pandas()` conversions on Dataset/EvaluationResult
`datacompy` -- Required by: `DataCompyScore` metric (`src/ragas/metrics/_datacompy_score.py:27-32`)

HuggingFace NLI

`transformers` -- Required by: `FaithfulnesswithHHEM` metric (HuggingFace NLI classifier, `src/ragas/metrics/_faithfulness.py:224-229`)

Statistical Analysis

`scikit-learn` -- Required by: `DiscreteMetric.correlation()` for Cohen's kappa (`src/ragas/metrics/discrete.py:82-88`), `RankingMetric.correlation()` (`src/ragas/metrics/ranking.py:83-89`)
`scipy` -- Required by: `NumericMetric.correlation()` for Pearson correlation (`src/ragas/metrics/numeric.py:80-86`)

Optimizer

`dspy-ai` >= 2.4.0 -- Required by: `DSPyOptimizer` for MIPROv2 prompt optimization (`src/ragas/optimizers/dspy_optimizer.py:77-85`). Install via `pip install "ragas[dspy]"`.

Version Control

`GitPython` -- Required by: `version_experiment()` for experiment git versioning (`src/ragas/experiment.py:45-57`). Install via `pip install "ragas[git]"`.

Tracing

`langfuse` >= 3.2.4 -- For Langfuse tracing integration. Install via `pip install "ragas[tracing]"`.
`mlflow` >= 3.1.4 -- For MLflow tracing integration. Install via `pip install "ragas[tracing]"`.

Credentials

`MLFLOW_HOST`: MLflow server URL, required when using the MLflow tracing integration.

Quick Install

# Install all common optional metrics packages
pip install "ragas[all]"

# Install specific packages as needed
pip install rouge_score sacrebleu rapidfuzz pandas datacompy

# Statistical analysis (for metric correlation)
pip install scikit-learn scipy

# DSPy optimizer
pip install "ragas[dspy]"

# Tracing integrations
pip install "ragas[tracing]"

# Git versioning for experiments
pip install "ragas[git]"

Code Evidence

ROUGE score import guard from `src/ragas/metrics/_rouge_score.py:21-27`:

try:
    from rouge_score import rouge_scorer
except ImportError:
    raise ImportError(
        "rouge_score is required for RougeScore metric. "
        "Please install it with: pip install rouge_score"
    )

Rapidfuzz import guard from `src/ragas/metrics/_string.py:70-75`:

try:
    from rapidfuzz import distance as rapidfuzz_distance
except ImportError:
    raise ImportError(
        "rapidfuzz is required for NonLLMStringSimilarity. "
        "Please install it with: pip install rapidfuzz"
    )

DSPy import guard from `src/ragas/optimizers/dspy_optimizer.py:77-85`:

try:
    import dspy
except ImportError:
    raise ImportError(
        "dspy is required for DSPyOptimizer. "
        "Install with: pip install 'ragas[dspy]' or pip install 'dspy-ai>=2.4.0'"
    )

Scikit-learn import guard from `src/ragas/metrics/discrete.py:82-88`:

try:
    from sklearn.metrics import cohen_kappa_score
except ImportError:
    raise ImportError(
        "scikit-learn is required for correlation computation. "
        "Please install it with: pip install scikit-learn"
    )

Common Errors

Error Message	Cause	Solution
`ImportError: rouge_score is required for RougeScore metric`	rouge_score not installed	`pip install rouge_score`
`ImportError: sacrebleu is required for BleuScore metric`	sacrebleu not installed	`pip install sacrebleu`
`ImportError: rapidfuzz is required for NonLLMStringSimilarity`	rapidfuzz not installed	`pip install rapidfuzz`
`ImportError: dspy is required for DSPyOptimizer`	dspy-ai not installed	`pip install "ragas[dspy]"`
`ImportError: scikit-learn is required for correlation`	scikit-learn not installed	`pip install scikit-learn`

Compatibility Notes

FaithfulnesswithHHEM: Defaults to CPU (`device="cpu"`). Pass `device="cuda"` for GPU acceleration when available.
All optional deps are lazy-loaded: Import errors only occur when you actually use the metric that needs the package, not at `import ragas` time.
`ragas[all]` extra: Installs `sentence-transformers`, `transformers`, `nltk`, `rouge_score`, `rapidfuzz`, `pandas`, `datacompy`, `sacrebleu`, `llama_index`, `r2r`, `GitPython`.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment