Implementation:Vibrantlabsai Ragas StringMetrics

Knowledge Sources	Vibrantlabsai_Ragas
Domains	Evaluation, Metrics
Last Updated	2026-02-12 00:00 GMT

Overview

The string metrics module provides three non-LLM-based metrics for comparing reference and response texts using exact matching, substring presence, and edit-distance-based similarity.

Description

This module defines a set of lightweight, deterministic string comparison metrics that do not require any LLM or embedding model. It contains three metric classes:

• ExactMatch -- Returns 1.0 if the response is exactly equal to the reference, and 0.0 otherwise. This is the simplest possible evaluation metric and is useful for tasks where the expected output is a precise string.

• StringPresence -- Returns 1.0 if the reference string appears as a substring within the response, and 0.0 otherwise. This is useful for checking whether a specific piece of information is present in a generated answer.

• NonLLMStringSimilarity -- Computes a normalized similarity score between the reference and response using classical string distance algorithms from the rapidfuzz library. The supported distance measures are defined in the DistanceMeasure enum: Levenshtein, Hamming, Jaro, and Jaro-Winkler. The score is computed as 1 - normalized_distance(reference, response), yielding a value between 0.0 and 1.0 where 1.0 indicates identical strings.

All three classes inherit from SingleTurnMetric and require the reference and response columns in the input sample.

Usage

Use ExactMatch when the expected output must be an exact string match (e.g., entity extraction, classification labels). Use StringPresence when you need to verify that a key piece of information appears somewhere in the response. Use NonLLMStringSimilarity when you want a graded similarity score between two strings without invoking an LLM, for example as a fast baseline metric or for string-level quality checks.

Code Reference

Source Location

• Repository: Vibrantlabsai_Ragas
• File: src/ragas/metrics/_string.py

Signature

class DistanceMeasure(Enum):
    LEVENSHTEIN = "levenshtein"
    HAMMING = "hamming"
    JARO = "jaro"
    JARO_WINKLER = "jaro_winkler"

@dataclass
class ExactMatch(SingleTurnMetric):
    name: str = "exact_match"
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {MetricType.SINGLE_TURN: {"reference", "response"}}
    )

@dataclass
class StringPresence(SingleTurnMetric):
    name: str = "string_present"
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {MetricType.SINGLE_TURN: {"reference", "response"}}
    )

@dataclass
class NonLLMStringSimilarity(SingleTurnMetric):
    name: str = "non_llm_string_similarity"
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {MetricType.SINGLE_TURN: {"reference", "response"}}
    )
    distance_measure: DistanceMeasure = DistanceMeasure.LEVENSHTEIN

Import

from ragas.metrics._string import ExactMatch, StringPresence, NonLLMStringSimilarity, DistanceMeasure

I/O Contract

Inputs (ExactMatch and StringPresence)

Name	Type	Required	Description
reference	str	Yes	The expected reference text
response	str	Yes	The generated response text to evaluate

Inputs (NonLLMStringSimilarity)

Name	Type	Required	Description
reference	str	Yes	The expected reference text
response	str	Yes	The generated response text to evaluate
distance_measure	DistanceMeasure	No	The string distance algorithm to use (default: LEVENSHTEIN). Options: LEVENSHTEIN, HAMMING, JARO, JARO_WINKLER

Outputs

Name	Type	Description
score	float	For ExactMatch: 1.0 if strings are identical, 0.0 otherwise. For StringPresence: 1.0 if reference is a substring of response, 0.0 otherwise. For NonLLMStringSimilarity: a float between 0.0 and 1.0 representing normalized similarity.

Usage Examples

ExactMatch

from ragas.metrics._string import ExactMatch
from ragas.dataset_schema import SingleTurnSample

metric = ExactMatch()

sample = SingleTurnSample(
    reference="Paris",
    response="Paris"
)
score = await metric._single_turn_ascore(sample, callbacks=None)
# score == 1.0

sample_mismatch = SingleTurnSample(
    reference="Paris",
    response="paris"
)
score = await metric._single_turn_ascore(sample_mismatch, callbacks=None)
# score == 0.0 (case-sensitive)

StringPresence

from ragas.metrics._string import StringPresence
from ragas.dataset_schema import SingleTurnSample

metric = StringPresence()

sample = SingleTurnSample(
    reference="Paris",
    response="The capital of France is Paris."
)
score = await metric._single_turn_ascore(sample, callbacks=None)
# score == 1.0

NonLLMStringSimilarity

from ragas.metrics._string import NonLLMStringSimilarity, DistanceMeasure
from ragas.dataset_schema import SingleTurnSample

# Using default Levenshtein distance
metric = NonLLMStringSimilarity()

sample = SingleTurnSample(
    reference="The capital of France is Paris.",
    response="The capital of France is Parris."
)
score = await metric._single_turn_ascore(sample, callbacks=None)
# score close to 1.0 (minor edit distance)

# Using Jaro-Winkler distance
metric_jw = NonLLMStringSimilarity(distance_measure=DistanceMeasure.JARO_WINKLER)
score_jw = await metric_jw._single_turn_ascore(sample, callbacks=None)

Related Pages

• Environment:Vibrantlabsai_Ragas_Python_3_9_Core_Environment