Implementation:Open compass VLMEvalKit VCRBench

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Video Comprehension

Overview

Benchmark dataset implementation for VCR-Bench video comprehension and reasoning evaluation in VLMEvalKit.

Description

VCRBench inherits from VideoBaseDataset and implements the VCR-Bench benchmark for evaluating video comprehension and reasoning capabilities. The TYPE field is set to 'Video-VQA'. It provides uniformly sampled frames in chronological order with system and frame template prompts for guiding model responses.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/vcrbench.py, Lines: L1-302
Import: from vlmeval.dataset.vcrbench import VCRBench

Signature:

class VCRBench(VideoBaseDataset):
    TYPE = 'Video-VQA'
    SYS = 'You are an AI assistant responsible for answering questions about videos.'
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with video paths and comprehension questions
Outputs	Evaluation results DataFrame with VQA scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('VCR-Bench')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment