Implementation:Explodinggradients Ragas ResponseRelevancy Metric

Field	Value
source	Repo
domains	Metrics, Evaluation
last_updated	2026-02-10

Overview

ResponseRelevancy (also aliased as AnswerRelevancy) scores the relevancy of an answer by generating questions from the response and computing cosine similarity between the generated questions and the original user input.

Description

The ResponseRelevancy class evaluates how relevant a response is to the user's question. It works by prompting an LLM to reverse-generate questions from the response, then computing cosine similarity between the embedding of the original question and the embeddings of the generated questions. Noncommittal answers (evasive, vague, or ambiguous) are penalized by zeroing the score. The metric inherits from MetricWithLLM, MetricWithEmbeddings, and SingleTurnMetric.

Key attributes:

• strictness -- Number of questions generated per answer (default 3, ideal range 3-5).
• question_generation -- The prompt used to generate questions from the response.

AnswerRelevancy is a subclass alias of ResponseRelevancy that provides backward compatibility.

Usage

The metric requires user_input and response columns. An LLM and embeddings must be configured.

Code Reference

Property	Value
Source Location	src/ragas/metrics/_answer_relevance.py L63-152
Class Signature	class ResponseRelevancy(MetricWithLLM, MetricWithEmbeddings, SingleTurnMetric)
Import	from ragas.metrics import AnswerRelevancy

I/O Contract

Inputs

Parameter	Type	Required	Description
user_input	str	Yes	The original question or user query
response	str	Yes	The generated answer to evaluate

Outputs

Output	Type	Description
score	float	Mean cosine similarity between original and generated questions (0.0 to 1.0), zeroed if noncommittal

Usage Examples

from ragas.metrics import AnswerRelevancy
from ragas.dataset_schema import SingleTurnSample

metric = AnswerRelevancy(strictness=3)
# metric.llm = ...  # Set your LLM
# metric.embeddings = ...  # Set your embeddings

sample = SingleTurnSample(
    user_input="Where was Albert Einstein born?",
    response="Albert Einstein was born in Germany."
)
# score = await metric.single_turn_ascore(sample)

A pre-configured instance is available:

from ragas.metrics._answer_relevance import answer_relevancy

Related Pages

• Explodinggradients_Ragas_SemanticSimilarity_Metric -- Embedding-based similarity metric
• Explodinggradients_Ragas_AnswerCorrectness_Metric -- Combines factuality with semantic similarity
• Explodinggradients_Ragas_Faithfulness_Metric -- Evaluates faithfulness of response to retrieved contexts