Implementation:Explodinggradients Ragas NoiseSensitivity Metric

Field	Value
source	Repo
domains	Metrics, Evaluation
last_updated	2026-02-10

Overview

NoiseSensitivity measures how much noise in the retrieved contexts impacts the correctness of a generated response, supporting both "relevant" and "irrelevant" noise modes.

Description

The NoiseSensitivity class evaluates whether incorrect statements in the response are caused by relevant or irrelevant noise in the retrieved contexts. It decomposes both the reference and the response into statements, then evaluates statement faithfulness against each retrieved context and the ground truth. The metric computes a score based on the proportion of incorrect answer statements that can be attributed to either relevant or irrelevant retrieved contexts. It inherits from MetricWithLLM and SingleTurnMetric.

Key attributes:

• mode -- Either "relevant" (default) to measure noise from relevant contexts or "irrelevant" for irrelevant contexts.
• nli_statements_prompt -- Prompt for NLI-based statement verification (reused from Faithfulness).
• statement_generator_prompt -- Prompt for decomposing text into statements (reused from Faithfulness).
• max_retries -- Maximum LLM retries (default 1).

Usage

The metric requires user_input, response, reference, and retrieved_contexts columns. An LLM must be configured.

Code Reference

Property	Value
Source Location	src/ragas/metrics/_noise_sensitivity.py L31-176
Class Signature	class NoiseSensitivity(MetricWithLLM, SingleTurnMetric)
Import	from ragas.metrics import NoiseSensitivity

I/O Contract

Inputs

Parameter	Type	Required	Description
user_input	str	Yes	The user query
response	str	Yes	The generated response to evaluate
reference	str	Yes	The ground truth reference answer
retrieved_contexts	List[str]	Yes	The retrieved context passages

Outputs

Output	Type	Description
score	float	Proportion of incorrect statements attributable to noise (0.0 to 1.0)

Usage Examples

from ragas.metrics import NoiseSensitivity
from ragas.dataset_schema import SingleTurnSample

metric = NoiseSensitivity(mode="relevant")
# metric.llm = ...  # Set your LLM

sample = SingleTurnSample(
    user_input="What powers the sun?",
    response="The sun is powered by nuclear fission.",
    reference="The sun is powered by nuclear fusion.",
    retrieved_contexts=[
        "Nuclear fusion powers the sun by fusing hydrogen atoms.",
        "Nuclear fission is used in power plants on Earth."
    ]
)
# score = await metric.single_turn_ascore(sample)

Related Pages

• Explodinggradients_Ragas_Faithfulness_Metric -- Shares statement decomposition and NLI prompts
• Explodinggradients_Ragas_FactualCorrectness_Metric -- Alternative factual accuracy metric
• Explodinggradients_Ragas_AnswerCorrectness_Metric -- Correctness metric combining factuality and similarity