Implementation:Open compass VLMEvalKit EgoExoBench MCQ

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Egocentric-Exocentric Video

Overview

Benchmark dataset implementation for EgoExoBench egocentric-exocentric video MCQ evaluation in VLMEvalKit.

Description

EgoExoBench_MCQ inherits from VideoBaseDataset and implements the EgoExoBench benchmark for evaluating multi-view video understanding from egocentric and exocentric perspectives. The TYPE field is set to 'Video-MCQ'. It downloads data from HuggingFace (Heleun/EgoExoBench_MCQ) with video content from a separate repository, supports 64-frame default sampling at 2 FPS, and includes an option to skip EgoExo4D subset.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/EgoExoBench/egoexobench.py, Lines: L1-298
Import: from vlmeval.dataset.EgoExoBench.egoexobench import EgoExoBench_MCQ

Signature:

class EgoExoBench_MCQ(VideoBaseDataset):
    TYPE = 'Video-MCQ'
    MD5 = '9c0aa8da235d766d02dd7e9a19182719'
    DEFAULT_JUDGE = ['chatgpt-0125', 'gpt-4-0125']
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with ego/exo video paths and MCQ questions
Outputs	Evaluation results DataFrame with MCQ accuracy scores per subtask

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('EgoExoBench_MCQ')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment