Implementation:Open compass VLMEvalKit MEGABench

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Multi-modal Video Understanding

Overview

Benchmark dataset implementation for MEGABench comprehensive multi-modal video evaluation in VLMEvalKit.

Description

MEGABench inherits from VideoBaseDataset and implements the MEGABench benchmark for large-scale multi-modal video understanding evaluation. The TYPE field is set to 'Video-VQA'. It supports subset selection (core subset by default), configurable frame sampling with max side length constraints, and handles both query and demo videos with separate frame allocation strategies.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/megabench.py, Lines: L1-494
Import: from vlmeval.dataset.megabench import MEGABench

Signature:

class MEGABench(VideoBaseDataset):
    TYPE = 'Video-VQA'
    ZIP_MD5 = '5ec01ab69cd25b643c4f5e1396e96441'
    MODALITY = 'VIDEO'
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with video paths and multi-modal understanding questions
Outputs	Evaluation results DataFrame with VQA scores per category

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('MEGABench')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment