Implementation:Open compass VLMEvalKit WorldSense

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Video World Understanding

Overview

Benchmark dataset implementation for WorldSense video world understanding evaluation in VLMEvalKit.

Description

WorldSense inherits from VideoBaseDataset and implements the WorldSense benchmark for evaluating video-based world understanding through multiple-choice questions. The TYPE field is set to 'Video-MCQ'. It supports multiple input modalities including frames without subtitles, frames with subtitles, and frames with audio, each with dedicated prompt templates guiding the model to respond with option letters (A, B, C, or D).

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/worldsense.py, Lines: L1-336
Import: from vlmeval.dataset.worldsense import WorldSense

Signature:

class WorldSense(VideoBaseDataset):
    TYPE = 'Video-MCQ'
    MD5 = 'bfc25490be4080aa5494b883370b6b1f'
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with video paths and world understanding MCQ questions
Outputs	Evaluation results DataFrame with MCQ accuracy scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('WorldSense')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment