Implementation:Open compass VLMEvalKit VisuLogic Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Visual Logic, Multiple Choice |
Overview
Provides answer extraction utilities for the VisuLogic visual logical reasoning benchmark with LaTeX boxed content parsing and multiple-choice option detection.
Description
This module implements `extract_last_boxed_content` for extracting answers from `\boxed{}` LaTeX expressions (with stack-based brace matching and LaTeX command cleanup), and `extract_lang_content` for detecting multiple-choice option letters (A-D) from various answer formats including parenthesized options, trailing periods, and keyword-based detection ("answer is", "correct answer"). It handles edge cases like multi-option answers and "LETTER" placeholder text.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/visulogic.py, Lines: L1-123 - Import:
from vlmeval.dataset.utils.visulogic import extract_last_boxed_content, extract_lang_content
Key Functions:
def extract_last_boxed_content(text): ...
def extract_lang_content(ans): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Model response text potentially containing `\boxed{}` expressions or multiple-choice indicators |
| Outputs | Extracted answer string (option letter or boxed content) |
Usage Examples
from vlmeval.dataset.utils.visulogic import extract_last_boxed_content
answer = extract_last_boxed_content("The answer is \\boxed{B}")
# answer = "B"
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment