Implementation:Explodinggradients Ragas Collections SemanticSimilarity Metric

Field	Value
source	Repo
domains	Metrics, Embeddings
last_updated	2026-02-10 00:00 GMT

Overview

SemanticSimilarity is a v2 class-based metric that evaluates semantic similarity between reference and response texts by computing the cosine similarity of their embedding vectors.

Description

SemanticSimilarity extends BaseMetric and requires a modern BaseRagasEmbedding instance. It is based on the Semantic Answer Similarity (SAS) approach described in the paper at https://arxiv.org/pdf/2108.06130.pdf. The implementation embeds both reference and response using the provided embeddings model, normalizes the resulting vectors, and computes their dot product to yield a cosine similarity score. An optional threshold parameter converts the continuous score into a binary classification (1.0 if similarity >= threshold, 0.0 otherwise). Empty inputs are treated as a single space to avoid division-by-zero errors.

Usage

Instantiate with a required embeddings parameter (a modern BaseRagasEmbedding instance) and an optional threshold. Call ascore(reference, response) for evaluation. The base class validates that the embeddings component is a modern implementation and rejects legacy wrappers.

Code Reference

Property	Value
Source Location	src/ragas/metrics/collections/_semantic_similarity.py L1--100
Signature	class SemanticSimilarity(BaseMetric)
Import	from ragas.metrics.collections import SemanticSimilarity

I/O Contract

Inputs

Parameter	Type	Required	Description
reference	str	Yes	The reference / ground truth text
response	str	Yes	The response text to evaluate

Constructor Parameters

Parameter	Type	Default	Description
embeddings	BaseRagasEmbedding	(required)	Modern embeddings model with embed_text() method
name	str	"semantic_similarity"	Metric name
threshold	Optional[float]	None	Optional threshold for binary classification

Outputs

Field	Type	Description
MetricResult.value	float	Cosine similarity score in range 0.0--1.0 (or binary 0.0/1.0 if threshold is set)

Usage Examples

from openai import AsyncOpenAI
from ragas.embeddings.base import embedding_factory
from ragas.metrics.collections import SemanticSimilarity

# Setup embeddings
client = AsyncOpenAI()
embeddings = embedding_factory(
    "openai", model="text-embedding-ada-002",
    client=client, interface="modern"
)

# Create metric
metric = SemanticSimilarity(embeddings=embeddings)

# Single evaluation
result = await metric.ascore(
    reference="Paris is the capital of France.",
    response="The capital of France is Paris."
)
print(f"Semantic Similarity: {result.value}")

# With binary threshold
binary_metric = SemanticSimilarity(embeddings=embeddings, threshold=0.8)
result = await binary_metric.ascore(
    reference="Hello world",
    response="Hi there world"
)
print(f"Similar enough: {result.value}")  # 1.0 or 0.0

# Batch evaluation
results = await metric.abatch_score([
    {"reference": "Text 1", "response": "Response 1"},
    {"reference": "Text 2", "response": "Response 2"},
])

Related Pages

• Explodinggradients_Ragas_Collections_BaseMetric_Class -- Base class for all v2 metrics
• Explodinggradients_Ragas_Collections_AnswerRelevancy_Metric -- Also uses embeddings for semantic comparison
• Explodinggradients_Ragas_Collections_StringMetrics_Module -- Non-embedding string similarity metrics