Implementation:Open compass VLMEvalKit MathV Utils

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Mathematics, Symbolic Comparison

Overview

Provides mathematical answer equivalence checking and GPT-based answer extraction for the MathV (Math Vision) benchmark.

Description

This module implements `is_equal` with a 30-second timeout decorator for comparing answers via direct string matching, numerical evaluation, and LaTeX symbolic parsing using `latex2sympy2`. The `get_gpt4_ICE` function provides in-context learning examples for GPT-4 based answer extraction from model responses. It uses tolerance-based comparison (1e-6) for numerical equivalence.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/mathv.py, Lines: L1-180
• Import: from vlmeval.dataset.utils.mathv import is_equal, get_gpt4_ICE

Key Functions:

@timeout_decorator.timeout(30)
def is_equal(asw: str, gt_asw: str) -> bool: ...
def get_gpt4_ICE(): ...

I/O Contract

Direction	Description
Inputs	Two answer strings (predicted and ground truth) for mathematical equivalence checking
Outputs	Boolean indicating whether the answers are equivalent

Usage Examples

from vlmeval.dataset.utils.mathv import is_equal

result = is_equal("0.5", "\\frac{1}{2}")
# result = True

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction