Implementation:Explodinggradients Ragas InstanceRubrics Metric

Field	Value
source	Repo
domains	Metrics, Evaluation
last_updated	2026-02-10

Overview

InstanceRubrics evaluates submissions using per-sample rubric definitions, allowing each evaluation instance to define its own scoring criteria.

Description

The InstanceRubrics class differs from RubricsScore in that rubrics are provided as part of each individual sample rather than being fixed at metric construction time. This allows for flexible, per-instance evaluation criteria. The rubrics dictionary is included in the prompt input alongside the standard fields. The class supports both single-turn and multi-turn evaluation. It inherits from MetricWithLLM, SingleTurnMetric, and MultiTurnMetric.

Key attributes:

• single_turn_prompt / multi_turn_prompt -- Prompts that accept rubrics as part of the input (using SingleTurnInputWithRubric / MultiTurnInputWithRubric models).
• max_retries -- Maximum LLM retries (default 1).

Usage

The metric requires a rubrics field in each sample. All other input columns are optional. An LLM must be configured.

Code Reference

Property	Value
Source Location	src/ragas/metrics/_instance_specific_rubrics.py L52-143
Class Signature	class InstanceRubrics(MetricWithLLM, SingleTurnMetric, MultiTurnMetric)
Import	from ragas.metrics import InstanceRubrics

I/O Contract

Inputs

Parameter	Type	Required	Description
rubrics	Dict[str, str]	Yes	Per-instance rubric definitions (e.g., score1_description through score5_description)
user_input	str	Optional	The user query or input
response	str	Optional	The generated response
retrieved_contexts	List[str]	Optional	Retrieved context passages
reference	str	Optional	The reference answer
reference_contexts	List[str]	Optional	Reference context passages

Outputs

Output	Type	Description
score	int	Discrete score based on the instance-specific rubric evaluation

Usage Examples

from ragas.metrics import InstanceRubrics
from ragas.dataset_schema import SingleTurnSample

metric = InstanceRubrics(name="custom_rubric")
# metric.llm = ...  # Set your LLM

sample = SingleTurnSample(
    user_input="Summarize the article.",
    response="The article discusses climate change impacts on agriculture.",
    rubrics={
        "score1_description": "Summary is completely inaccurate.",
        "score2_description": "Summary captures some points but misses key details.",
        "score3_description": "Summary is adequate but lacks depth.",
        "score4_description": "Summary is accurate and covers most key points.",
        "score5_description": "Summary is comprehensive, accurate, and well-structured."
    }
)
# score = await metric.single_turn_ascore(sample)

Related Pages

• Implementation:Explodinggradients_Ragas_RubricsScore_Metric -- Fixed rubric evaluation (rubrics set at construction time)
• Implementation:Explodinggradients_Ragas_SimpleCriteriaScore_Metric -- User-defined criteria evaluation
• Implementation:Explodinggradients_Ragas_AspectCritic_Metric -- Binary aspect-based evaluation