Implementation:Explodinggradients Ragas Collections CHRFScore Metric

Field	Value
source	Repo
domains	Metrics, NLP
last_updated	2026-02-10 00:00 GMT

Overview

CHRFScore is a v2 class-based metric that calculates the CHRF (Character n-gram F-score) between reference and response texts using the sacrebleu library.

Description

CHRFScore extends BaseMetric to provide an async-first character-level n-gram F-score implementation. Unlike BLEU which operates on word-level n-grams, CHRF uses character-level n-grams, making it more robust to morphological variations and better suited for morphologically rich languages. The metric correlates well with human judgments for machine translation quality.

The implementation uses sacrebleu.corpus_chrf and normalizes the raw score (0--100) to a 0.0--1.0 range. It includes built-in input validation, returning a MetricResult with value 0.0 and an explanatory reason for non-string or empty inputs. No LLM or embedding components are required.

Usage

Instantiate with an optional name and kwargs dict (forwarded to sacrebleu.corpus_chrf, supporting parameters like char_order, word_order, beta, and eps_smoothing). Call ascore(reference, response) for single evaluations. Requires the sacrebleu package.

Code Reference

Property	Value
Source Location	`src/ragas/metrics/collections/chrf_score/metric.py` L1--96
Signature	`class CHRFScore(BaseMetric)`
Import	`from ragas.metrics.collections import CHRFScore`

I/O Contract

Inputs

Parameter	Type	Required	Description
`reference`	`str`	Yes	The reference / ground truth text
`response`	`str`	Yes	The response text to evaluate

Constructor Parameters

Parameter	Type	Default	Description
`name`	`str`	`"chrf_score"`	Metric name
`kwargs`	`Optional[Dict[str, Any]]`	`None`	Extra arguments forwarded to `sacrebleu.corpus_chrf`

Outputs

Field	Type	Description
`MetricResult.value`	`float`	CHRF score in range 0.0--1.0
`MetricResult.reason`	`Optional[str]`	Explanation when returning 0.0 due to invalid/empty input

Usage Examples

from ragas.metrics.collections import CHRFScore

# Basic usage
metric = CHRFScore()
result = await metric.ascore(
    reference="The capital of France is Paris.",
    response="Paris is the capital of France."
)
print(f"CHRF Score: {result.value}")

# With custom sacrebleu parameters
metric = CHRFScore(kwargs={"char_order": 6, "word_order": 2, "beta": 2})
result = await metric.ascore(
    reference="The quick brown fox jumps over the lazy dog.",
    response="A quick brown fox jumped over the lazy dog."
)
print(f"Custom CHRF Score: {result.value}")

# Batch evaluation
results = await metric.abatch_score([
    {"reference": "Text one.", "response": "Response one."},
    {"reference": "Text two.", "response": "Response two."},
])
for r in results:
    print(r.value)

# Edge case: empty input returns 0.0 with reason
result = await metric.ascore(reference="", response="Some text")
print(f"Score: {result.value}, Reason: {result.reason}")

Related Pages

Explodinggradients_Ragas_Collections_BaseMetric_Class -- Base class for all v2 metrics
Explodinggradients_Ragas_Collections_BleuScore_Metric -- Word-level n-gram BLEU metric (also uses sacrebleu)
Explodinggradients_Ragas_Collections_RougeScore_Metric -- ROUGE recall-oriented metric
Explodinggradients_Ragas_Collections_StringMetrics_Module -- Other non-LLM string comparison metrics

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment