Implementation:Open compass VLMEvalKit GSM8K V Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Mathematics, Grade School Math |
Overview
Provides numerical answer extraction and comparison utilities for the GSM8K-V (visual grade school math) benchmark evaluation.
Description
This module implements robust numerical answer extraction from model responses using multiple regex patterns including "FINAL ANSWER", \boxed{}, "####", and "The answer is" formats. The `extract_numerical_answer` function normalizes extracted values to floats via `_normalize_to_float`, handling fractions and comma-separated numbers. The `_clean_extracted_number` function validates and sanitizes number strings before conversion.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/gsm8k_v.py, Lines: L1-228 - Import:
from vlmeval.dataset.utils.gsm8k_v import extract_numerical_answer
Key Functions:
def extract_numerical_answer(text): ...
def _clean_extracted_number(number_str): ...
def _get_last_paragraph(text): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Model response text string containing a numerical answer |
| Outputs | Extracted float value or None if no valid numerical answer found |
Usage Examples
from vlmeval.dataset.utils.gsm8k_v import extract_numerical_answer
answer = extract_numerical_answer("The answer is 42.5")
# answer = 42.5
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment