Implementation:Open compass VLMEvalKit MathVerse Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Mathematics, Answer Extraction |
Overview
Provides GPT-based answer extraction and scoring utilities for the MathVerse visual mathematics benchmark.
Description
This module implements `get_gpt4_extract_ICE` and `get_gpt4_score_ICE` which provide in-context examples for two-stage evaluation: first extracting answers from model responses (supporting coordinates, option letters, intervals, and equations), then scoring the extracted answers against standard answers. The scoring stage uses six detailed examples covering various mathematical answer types including intervals, multiple choice, and free-form expressions.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/mathverse.py, Lines: L1-193 - Import:
from vlmeval.dataset.utils.mathverse import get_gpt4_extract_ICE, get_gpt4_score_ICE
Key Functions:
def get_gpt4_extract_ICE(): ...
def get_gpt4_score_ICE(): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Model response strings requiring answer extraction and scoring |
| Outputs | Lists of in-context example strings for GPT-based extraction and scoring |
Usage Examples
from vlmeval.dataset.utils.mathverse import get_gpt4_extract_ICE
examples = get_gpt4_extract_ICE()
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment