Implementation:Explodinggradients Ragas AspectCritic Metric

Field	Value
source	Repo
domains	Metrics, Evaluation
last_updated	2026-02-10

Overview

AspectCritic provides binary (0/1) evaluation of submissions using user-defined aspect criteria, with pre-configured instances for harmfulness, maliciousness, coherence, correctness, and conciseness.

Description

The AspectCritic class judges submissions against a user-defined criteria definition, returning a binary verdict. It supports both single-turn and multi-turn evaluation. A self-consistency mechanism via the strictness parameter runs multiple checks and uses majority voting to determine the final verdict. The class inherits from MetricWithLLM, SingleTurnMetric, and MultiTurnMetric.

Key attributes:

• definition -- The criteria string used to evaluate submissions (e.g., "Is the submission spreading fake information?").
• strictness -- Number of self-consistency checks (forced to odd to avoid ties, default 1).
• single_turn_prompt / multi_turn_prompt -- Customizable prompts for each evaluation mode.

Pre-configured instances: harmfulness, maliciousness, coherence, correctness, conciseness.

Usage

All input columns are optional. The metric requires an LLM and a definition string to be provided at construction time.

Code Reference

Property	Value
Source Location	src/ragas/metrics/_aspect_critic.py L75-242
Class Signature	class AspectCritic(MetricWithLLM, SingleTurnMetric, MultiTurnMetric)
Import	from ragas.metrics import AspectCritic

I/O Contract

Inputs

Parameter	Type	Required	Description
user_input	str	Optional	The user query or input
response	str	Optional	The generated response
retrieved_contexts	List[str]	Optional	Retrieved context passages
reference	str	Optional	The reference answer
reference_contexts	List[str]	Optional	Reference context passages

Outputs

Output	Type	Description
score	float	Binary verdict: 0 (criteria not met) or 1 (criteria met)

Usage Examples

from ragas.metrics import AspectCritic
from ragas.dataset_schema import SingleTurnSample

# Custom aspect critic
critic = AspectCritic(
    name="factual_accuracy",
    definition="Is the submission factually accurate and free from errors?",
    strictness=3
)
# critic.llm = ...  # Set your LLM

sample = SingleTurnSample(
    user_input="What is the capital of France?",
    response="The capital of France is Paris."
)
# score = await critic.single_turn_ascore(sample)

Pre-configured instances:

from ragas.metrics._aspect_critic import harmfulness, maliciousness, coherence, correctness, conciseness

Related Pages

• Explodinggradients_Ragas_SimpleCriteriaScore_Metric -- Similar criteria-based evaluation but with discrete (non-binary) scoring
• Explodinggradients_Ragas_RubricsScore_Metric -- Rubric-based evaluation with 1-5 scoring
• Explodinggradients_Ragas_Faithfulness_Metric -- Statement-level faithfulness evaluation