Implementation:Open compass VLMEvalKit MMMath

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Mathematical Reasoning

Overview

Benchmark dataset implementation for MM-Math multimodal mathematical reasoning evaluation in VLMEvalKit.

Description

MMMath inherits from ImageBaseDataset and implements the MM-Math benchmark for evaluating mathematical reasoning capabilities of multimodal models. The TYPE field is set to 'VQA'. The file also defines an AutoScoringJudge class for automated answer verification including symbolic math expression comparison with support for power expressions, fractions, and LaTeX notation.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/mmmath.py, Lines: L1-459
Import: from vlmeval.dataset.mmmath import MMMath

Signature:

class MMMath(ImageBaseDataset):
    TYPE = 'VQA'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with math problem images and questions
Outputs	Evaluation results DataFrame with mathematical reasoning accuracy scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('MM-Math')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment