Implementation:Open compass VLMEvalKit LENS Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Mathematics, Symbolic |
Overview
Provides answer equivalence checking utilities for the LENS benchmark using symbolic mathematics (latex2sympy) and numerical comparison.
Description
This module implements the `is_equal` function that compares predicted and ground truth answers through multiple strategies: direct string comparison, numerical evaluation via `eval()`, and symbolic LaTeX parsing via `latex2sympy`. It also provides `get_gpt4_ICE` for constructing in-context examples used in GPT-4 based answer extraction prompts. The module supports both latex2sympy2-extended and latex2sympy2 backends.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/lens.py, Lines: L1-215 - Import:
from vlmeval.dataset.utils.lens import is_equal, get_gpt4_ICE
Key Functions:
def is_equal(asw: str, gt_asw: str) -> bool: ...
def get_gpt4_ICE(): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Two answer strings (predicted and ground truth) for comparison |
| Outputs | Boolean indicating whether the answers are mathematically equivalent |
Usage Examples
from vlmeval.dataset.utils.lens import is_equal
result = is_equal("3.14", "\\pi")
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment