Implementation:Open compass VLMEvalKit MMSIVideoBench

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Video Scene Interaction

Overview

Benchmark dataset implementation for MMSIVideo evaluation in VLMEvalKit.

Description

MMSIVideoBench inherits from ImageBaseDataset and implements the MMSI Video benchmark for multi-modal scene interaction evaluation in video. The TYPE field is set to 'MCQ'. It covers categories including cross-video memory update, multi-view integration, planning, prediction, and motion understanding.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/mmsi_video.py, Lines: L1-297
Import: from vlmeval.dataset.mmsi_video import MMSIVideoBench

Signature:

class MMSIVideoBench(ImageBaseDataset):
    TYPE = 'MCQ'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with image/video paths and questions
Outputs	Evaluation results DataFrame with scores per category

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('MMSIVideo_U50')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment