Implementation:Open compass VLMEvalKit MathVista Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Mathematics, Multi-format |
Overview
Provides GPT-based answer extraction utilities for the MathVista benchmark, supporting integers, floats, lists, and multiple-choice answers.
Description
This module implements `get_gpt4_ICE` which provides five in-context examples covering different answer types: integers, single-decimal floats, two-decimal floats, Python lists, and multiple-choice options. The `build_mathvista_gpt4_prompt` function constructs extraction prompts that guide GPT-4 to extract answers from model responses based on the question's hint about expected answer format.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/mathvista.py, Lines: L1-164 - Import:
from vlmeval.dataset.utils.mathvista import build_mathvista_gpt4_prompt, get_gpt4_ICE
Key Functions:
def get_gpt4_ICE(): ...
def build_mathvista_gpt4_prompt(line): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | A data line dict with 'question' and 'prediction' fields |
| Outputs | Formatted GPT-4 prompt string for answer extraction |
Usage Examples
from vlmeval.dataset.utils.mathvista import build_mathvista_gpt4_prompt
prompt = build_mathvista_gpt4_prompt(line)
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment