Implementation:Open compass VLMEvalKit MEGABench Answer Str Parse

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Answer Parsing, Text Extraction

Overview

Parses model response strings to extract structured answers for the MEGA-Bench evaluation framework.

Description

The `AnswerStrParse` class implements a multi-strategy answer extraction pipeline. It first extracts content from "Answer: ..." formatted responses via `extract_answer_content`, then processes code blocks via `extract_code_block_content`, and handles single-line answers with `drop_additional_text`. For structured answers, it attempts JSON parsing via `parse_json`. The parser supports ASCII art preservation and configurable whitespace handling through parameters like `is_ascii_art` and `should_remove_surrounding_whitespace`.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/megabench/parsing/answer_str_parse.py, Lines: L1-137
• Import: from vlmeval.dataset.utils.megabench.parsing.answer_str_parse import AnswerStrParse

Key Functions:

class AnswerStrParse:
    @classmethod
    def _parse(cls, response, *, is_ascii_art=False, ...): ...

I/O Contract

Direction	Description
Inputs	Raw model response string, optional flags for ASCII art mode and whitespace handling
Outputs	Parsed answer as string, dict, or list depending on the response content structure

Usage Examples

from vlmeval.dataset.utils.megabench.parsing.answer_str_parse import AnswerStrParse

result = AnswerStrParse._parse("Answer: 42")

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction