Implementation:Explodinggradients Ragas LLMSQLEquivalence Metric

Field	Value
source	Repo
domains	Metrics, SQL
last_updated	2026-02-10

Overview

LLMSQLEquivalence uses an LLM to determine whether a generated SQL query is semantically equivalent to a reference SQL query, given a database schema.

Description

The LLMSQLEquivalence class evaluates SQL generation quality by comparing the logical equivalence of two SQL queries. The LLM is provided with both the reference and generated SQL queries along with the database schema, and asked to explain each query then determine if they are equivalent. The output model includes explanations of both queries and a boolean equivalence verdict. It inherits from MetricWithLLM and SingleTurnMetric.

Key attributes:

equivalence_prompt -- The prompt used for SQL comparison (default EquivalencePrompt).

Usage

The metric requires response (generated SQL), reference (ground truth SQL), and reference_contexts (database schema as a list of strings). An LLM must be configured.

Code Reference

Property	Value
Source Location	`src/ragas/metrics/_sql_semantic_equivalence.py` L70-104
Class Signature	`class LLMSQLEquivalence(MetricWithLLM, SingleTurnMetric)`
Import	`from ragas.metrics import LLMSQLEquivalence`

I/O Contract

Inputs

Parameter	Type	Required	Description
response	str	Yes	The generated SQL query
reference	str	Yes	The ground truth SQL query
reference_contexts	List[str]	Yes	The database schema definition(s)

Outputs

Output	Type	Description
score	float	Binary: 1.0 if queries are semantically equivalent, 0.0 otherwise

Usage Examples

from ragas.metrics import LLMSQLEquivalence
from ragas.dataset_schema import SingleTurnSample

metric = LLMSQLEquivalence()
# metric.llm = ...  # Set your LLM

sample = SingleTurnSample(
    response="SELECT id, name FROM users WHERE active = true;",
    reference="SELECT id, name FROM users WHERE active = 1;",
    reference_contexts=[
        "Table users: id INT, name VARCHAR, active BOOLEAN"
    ]
)
# score = await metric.single_turn_ascore(sample)

Related Pages

Explodinggradients_Ragas_FactualCorrectness_Metric -- Factual correctness evaluation for natural language
Explodinggradients_Ragas_AspectCritic_Metric -- Binary evaluation with custom criteria
Explodinggradients_Ragas_StringMetrics_Module -- Non-LLM string comparison metrics

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment