| Field |
Value
|
| source |
Repo
|
| domains |
Metrics, Embeddings
|
| last_updated |
2026-02-10 00:00 GMT
|
Overview
SemanticSimilarity is a v2 class-based metric that evaluates semantic similarity between reference and response texts by computing the cosine similarity of their embedding vectors.
Description
SemanticSimilarity extends BaseMetric and requires a modern BaseRagasEmbedding instance. It is based on the Semantic Answer Similarity (SAS) approach described in the paper at https://arxiv.org/pdf/2108.06130.pdf. The implementation embeds both reference and response using the provided embeddings model, normalizes the resulting vectors, and computes their dot product to yield a cosine similarity score. An optional threshold parameter converts the continuous score into a binary classification (1.0 if similarity >= threshold, 0.0 otherwise). Empty inputs are treated as a single space to avoid division-by-zero errors.
Usage
Instantiate with a required embeddings parameter (a modern BaseRagasEmbedding instance) and an optional threshold. Call ascore(reference, response) for evaluation. The base class validates that the embeddings component is a modern implementation and rejects legacy wrappers.
Code Reference
| Property |
Value
|
| Source Location |
src/ragas/metrics/collections/_semantic_similarity.py L1--100
|
| Signature |
class SemanticSimilarity(BaseMetric)
|
| Import |
from ragas.metrics.collections import SemanticSimilarity
|
I/O Contract
Inputs
| Parameter |
Type |
Required |
Description
|
reference |
str |
Yes |
The reference / ground truth text
|
response |
str |
Yes |
The response text to evaluate
|
Constructor Parameters
| Parameter |
Type |
Default |
Description
|
embeddings |
BaseRagasEmbedding |
(required) |
Modern embeddings model with embed_text() method
|
name |
str |
"semantic_similarity" |
Metric name
|
threshold |
Optional[float] |
None |
Optional threshold for binary classification
|
Outputs
| Field |
Type |
Description
|
MetricResult.value |
float |
Cosine similarity score in range 0.0--1.0 (or binary 0.0/1.0 if threshold is set)
|
Usage Examples
from openai import AsyncOpenAI
from ragas.embeddings.base import embedding_factory
from ragas.metrics.collections import SemanticSimilarity
# Setup embeddings
client = AsyncOpenAI()
embeddings = embedding_factory(
"openai", model="text-embedding-ada-002",
client=client, interface="modern"
)
# Create metric
metric = SemanticSimilarity(embeddings=embeddings)
# Single evaluation
result = await metric.ascore(
reference="Paris is the capital of France.",
response="The capital of France is Paris."
)
print(f"Semantic Similarity: {result.value}")
# With binary threshold
binary_metric = SemanticSimilarity(embeddings=embeddings, threshold=0.8)
result = await binary_metric.ascore(
reference="Hello world",
response="Hi there world"
)
print(f"Similar enough: {result.value}") # 1.0 or 0.0
# Batch evaluation
results = await metric.abatch_score([
{"reference": "Text 1", "response": "Response 1"},
{"reference": "Text 2", "response": "Response 2"},
])
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
