Implementation:Open compass VLMEvalKit QBench Video
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Benchmarking, Video Quality Assessment |
Overview
Benchmark dataset implementation for Q-Bench Video quality assessment evaluation in VLMEvalKit.
Description
QBench_Video inherits from ConcatVideoDataset and aggregates QBench_Video_MCQ and QBench_Video_VQA sub-datasets. The file defines three classes: QBench_Video (the concatenated dataset), QBench_Video_MCQ (TYPE 'Video-MCQ') for multiple-choice quality assessment questions, and QBench_Video_VQA (TYPE 'Video-VQA') for open-ended quality evaluation. The open-ended score is halved in the final evaluation.
Usage
Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.
Code Reference
- Source:
vlmeval/dataset/qbench_video.py, Lines: L1-353 - Import:
from vlmeval.dataset.qbench_video import QBench_Video
Signature:
class QBench_Video(ConcatVideoDataset):
...
class QBench_Video_MCQ(VideoBaseDataset):
TYPE = 'Video-MCQ'
...
class QBench_Video_VQA(VideoBaseDataset):
TYPE = 'Video-VQA'
...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | TSV dataset file with video paths and quality assessment questions |
| Outputs | Evaluation results DataFrame with MCQ and VQA quality scores |
Usage Examples
from vlmeval.dataset import build_dataset
dataset = build_dataset('QBench_Video')
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment