Implementation:Open compass VLMEvalKit DREAM

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Video Understanding

Overview

Benchmark dataset implementation for DREAM-1K video understanding evaluation in VLMEvalKit.

Description

DREAM inherits from VideoBaseDataset and implements the DREAM-1K benchmark for evaluating video understanding capabilities. The TYPE field is set to 'DREAM-1K'. It downloads data from HuggingFace (mjuicem/DREAM-1k-VLMEvalKit) and defaults to 8 frames when no frame count or FPS is specified. Video files are expected as MP4 format.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/dream.py, Lines: L1-402
Import: from vlmeval.dataset.dream import DREAM

Signature:

class DREAM(VideoBaseDataset):
    TYPE = 'DREAM-1K'
    MD5 = 'e8f0a486429bb6c27806bc0669e0d8b2'
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with video paths and understanding questions
Outputs	Evaluation results DataFrame with scores per category

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('DREAM-1K')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment