Implementation:Open compass VLMEvalKit BMMR Utils

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Mathematics, Bilingual

Overview

Provides utility functions for the BMMR (Bilingual Multimodal Mathematical Reasoning) benchmark evaluation, including answer extraction and mathematical equivalence checking.

Description

This module implements answer extraction from LaTeX-formatted model responses using `extract_boxed_content` (parsing \boxed{} expressions), `extract_text` (parsing \text{} commands), and `extract_uppercase` (extracting option letters). It uses the `math_equal` function from `bmmr_grade` for mathematical equivalence comparison. The module integrates with the HuggingFace `evaluate` library and supports BLEU-based scoring alongside exact mathematical matching.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/bmmr.py, Lines: L1-281
• Import: from vlmeval.dataset.utils.bmmr import extract_boxed_content, extract_text

Key Functions:

def extract_boxed_content(text): ...
def extract_text(input_string): ...
def extract_uppercase(s): ...

I/O Contract

Direction	Description
Inputs	Model response strings containing LaTeX-formatted answers (e.g., \boxed{42})
Outputs	Extracted answer strings or lists for downstream comparison

Usage Examples

from vlmeval.dataset.utils.bmmr import extract_boxed_content

answers = extract_boxed_content("The answer is \\boxed{42}")
# answers = ['42']

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction