Implementation:Open compass VLMEvalKit HiPhODataset

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Physics Olympiad

Overview

Benchmark dataset implementation for HiPhO (High School Physics Olympiad) evaluation in VLMEvalKit.

Description

HiPhODataset inherits from ImageBaseDataset and implements the HiPhO benchmark for evaluating multimodal models on physics olympiad competition problems. The TYPE field is set to 'VQA'. It supports 13 physics competition datasets including IPhO, EuPhO, APhO, PanPhO, NBPhO, F_MA, and PanMechanics across multiple years (2024/2025), with integrated hipho_verifier for fine and coarse-grained evaluation.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/hipho.py, Lines: L1-920
Import: from vlmeval.dataset.hipho import HiPhODataset

Signature:

class HiPhODataset(ImageBaseDataset):
    TYPE = 'VQA'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with physics problem images and questions
Outputs	Evaluation results DataFrame with fine/coarse-grained accuracy scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('IPhO_2024')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment