Implementation:Open compass VLMEvalKit MMHelix Parser

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Answer Parsing, Multi-format

Overview

Provides a default answer parser for the MMHelix benchmark that extracts answers from various formats including boxed LaTeX, answer tags, and matrix notations.

Description

The `DefaultParser` class implements a multi-strategy parsing pipeline that attempts extraction in order: `<|begin_of_box|>...<|end_of_box|>` tags, `<answer>...</answer>` tags, `\boxed{}` LaTeX format (with nested bracket support via the `regex` module), `\text{}` within boxed content, `\begin{array}...\end{array}`, and `\begin{bmatrix}...\end{bmatrix}`. It always takes the last match to handle cases where models revise their answers. The parser also handles truncated `\text{}` variants.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/mmhelix/parser.py, Lines: L1-71
• Import: from vlmeval.dataset.utils.mmhelix.parser import DefaultParser

Key Functions:

class DefaultParser:
    def parse(self, response): ...

I/O Contract

Direction	Description
Inputs	Raw model response string potentially containing boxed LaTeX, answer tags, or matrix notations
Outputs	Extracted answer string from the last matching format found

Usage Examples

from vlmeval.dataset.utils.mmhelix.parser import DefaultParser

parser = DefaultParser()
answer = parser.parse("The answer is \\boxed{42}")
# answer = "42"

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction