Implementation:Explodinggradients Ragas ContextPrecision Metric

Field	Value
source	Repo
domains	Metrics, Evaluation
last_updated	2026-02-10

Overview

ContextPrecision evaluates whether relevant items in the retrieved context are ranked higher, using Average Precision with multiple implementation variants (LLM-based, non-LLM, and ID-based).

Description

The context precision module provides several classes for evaluating retrieval quality:

LLMContextPrecisionWithReference -- Uses an LLM to verify if each retrieved context was useful in arriving at the reference answer, then computes Average Precision over the ranked list.
LLMContextPrecisionWithoutReference -- Same as above but uses the response instead of a reference answer.
NonLLMContextPrecisionWithReference -- Uses non-LLM string similarity to compare retrieved contexts against reference contexts.
IDBasedContextPrecision -- Directly compares retrieved context IDs with reference context IDs for precision.
ContextPrecision -- Convenience alias for LLMContextPrecisionWithReference.
ContextUtilization -- Convenience alias for LLMContextPrecisionWithoutReference.

The LLM-based variants use an ensembler for aggregating multiple verification responses. The non-LLM variant uses a configurable distance measure with a threshold for binary relevance decisions.

Usage

Required columns vary by variant. LLM-based variants require user_input, retrieved_contexts, and either reference or response. Non-LLM requires retrieved_contexts and reference_contexts. ID-based requires retrieved_context_ids and reference_context_ids.

Code Reference

Property	Value
Source Location	`src/ragas/metrics/_context_precision.py` L81-331
Class Signature	`class LLMContextPrecisionWithReference(MetricWithLLM, SingleTurnMetric)`
Import	`from ragas.metrics import ContextPrecision`

I/O Contract

Inputs (LLMContextPrecisionWithReference / ContextPrecision)

Parameter	Type	Required	Description
user_input	str	Yes	The user query
retrieved_contexts	List[str]	Yes	The ranked list of retrieved context passages
reference	str	Yes	The ground truth reference answer

Outputs

Output	Type	Description
score	float	Average Precision score (0.0 to 1.0)

Usage Examples

from ragas.metrics import ContextPrecision
from ragas.dataset_schema import SingleTurnSample

metric = ContextPrecision()
# metric.llm = ...  # Set your LLM

sample = SingleTurnSample(
    user_input="What is the capital of France?",
    retrieved_contexts=[
        "Paris is the capital of France.",
        "The Eiffel Tower is in Paris.",
        "France is in Europe."
    ],
    reference="The capital of France is Paris."
)
# score = await metric.single_turn_ascore(sample)

Pre-configured instances:

from ragas.metrics._context_precision import context_precision, context_utilization

Related Pages

Explodinggradients_Ragas_ContextRecall_Metric -- Measures context recall (complementary metric)
Explodinggradients_Ragas_ContextEntityRecall_Metric -- Entity-level context recall
Explodinggradients_Ragas_StringMetrics_Module -- NonLLMStringSimilarity used by the non-LLM variant

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment