Implementation:Open compass VLMEvalKit LongVideoBench
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Benchmarking, Long Video Understanding |
Overview
Benchmark dataset implementation for LongVideoBench long video multiple-choice evaluation in VLMEvalKit.
Description
LongVideoBench inherits from VideoBaseDataset and implements the LongVideoBench benchmark for evaluating models on long video understanding through multiple-choice questions. The TYPE field is set to 'Video-MCQ'. It supports optional subtitle usage and downloads data from HuggingFace (longvideobench/LongVideoBench) with MD5 integrity verification.
Usage
Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.
Code Reference
- Source:
vlmeval/dataset/longvideobench.py, Lines: L1-332 - Import:
from vlmeval.dataset.longvideobench import LongVideoBench
Signature:
class LongVideoBench(VideoBaseDataset):
MD5 = '82905eae3a5ae7383c5a8ee9655e1ab9'
TYPE = 'Video-MCQ'
DEFAULT_JUDGE = ['chatgpt-0125', 'gpt-4-0125']
...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | TSV dataset file with long video paths and multiple-choice questions |
| Outputs | Evaluation results DataFrame with MCQ accuracy scores |
Usage Examples
from vlmeval.dataset import build_dataset
dataset = build_dataset('LongVideoBench')
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment