Implementation:Explodinggradients Ragas ChrfScore Metric

Field	Value
source	Repo
domains	Metrics, NLP
last_updated	2026-02-10

Overview

ChrfScore computes the chrF (character n-gram F-score) between a reference and a generated response using the sacrebleu library.

Description

The ChrfScore class evaluates text generation quality by computing the chrF score, which measures character-level n-gram overlap between reference and response. Unlike BLEU which operates on word-level n-grams, chrF uses character-level n-grams and is more robust across languages. The score is normalized to a 0-1 range by dividing by 100. The metric handles edge cases like None values, non-string inputs, and empty strings by returning 0.0. It inherits only from SingleTurnMetric.

Key attributes:

• kwargs -- Additional keyword arguments passed to the underlying corpus_chrf function.

Dependency: Requires the sacrebleu package (pip install sacrebleu).

Usage

The metric requires reference and response columns.

Code Reference

Property	Value
Source Location	src/ragas/metrics/_chrf_score.py L11-56
Class Signature	class ChrfScore(SingleTurnMetric)
Import	from ragas.metrics import ChrfScore

I/O Contract

Inputs

Parameter	Type	Required	Description
reference	str	Yes	The ground truth reference text
response	str	Yes	The generated response to evaluate

Outputs

Output	Type	Description
score	float	chrF score normalized to 0.0-1.0 range

Usage Examples

from ragas.metrics import ChrfScore
from ragas.dataset_schema import SingleTurnSample

metric = ChrfScore()

sample = SingleTurnSample(
    reference="The quick brown fox jumps over the lazy dog.",
    response="The fast brown fox leaps over the lazy dog."
)
# score = await metric.single_turn_ascore(sample)

Related Pages

• Implementation:Explodinggradients_Ragas_BleuScore_Metric -- BLEU-based word n-gram precision metric
• Implementation:Explodinggradients_Ragas_RougeScore_Metric -- ROUGE-based n-gram overlap metric
• Implementation:Explodinggradients_Ragas_StringMetrics_Module -- Non-LLM string comparison metrics