Implementation:Open compass VLMEvalKit SeePhys Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Physics, Answer Grading |
Overview
Provides physics answer evaluation with LLM-as-judge for the SeePhys (See Physics) benchmark, supporting both English and Chinese physics questions with extensive prompt engineering.
Description
This module implements prompt_scoring containing detailed evaluation prompts for physics answer equivalence checking. The prompt includes 11 graded examples covering mechanics, projectile motion, electromagnetism, relativistic physics, and quantum mechanics. It handles unit equivalence (kW vs watts), LaTeX expression comparison, trigonometric identity equivalence, and multi-part answer verification. The scoring system uses a binary (0/1) judgement format, and the prompts support both English and Chinese physics problems with mathematical notation in LaTeX format.
Usage
Called internally by the SeePhys dataset class during physics answer evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/seephys.py, Lines: L1-312 - Import:
from vlmeval.dataset.utils.seephys import prompt_scoring
Key Functions:
prompt_scoring = r"""...""" # Detailed physics scoring prompt with 11 examples
def evaluate_answer(prediction, reference, question, model): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Predicted physics answer, standard answer, original question text, and LLM judge model |
| Outputs | Binary judgement (0 or 1) indicating whether the predicted answer is equivalent to the standard answer |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.seephys import prompt_scoring
# Use prompt_scoring template with question, standard answer, and model answer