Implementation:Open compass VLMEvalKit MVTamperBench

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Video Tampering Detection

Overview

Benchmark dataset implementation for MVTamperBench video tampering detection evaluation in VLMEvalKit.

Description

MVTamperBench inherits from VideoBaseDataset and implements the MVTamperBench benchmark for evaluating models on video tampering detection through multiple-choice questions. The TYPE field is set to 'Video-MCQ'. It supports three variants (MVTamperBench, MVTamperBenchStart, MVTamperBenchEnd) each with separate MD5 checksums, and covers temporal reasoning tasks similar to MVBench including action sequences, predictions, and scene transitions.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/tamperbench.py, Lines: L1-534
Import: from vlmeval.dataset.tamperbench import MVTamperBench

Signature:

class MVTamperBench(VideoBaseDataset):
    TYPE = 'Video-MCQ'
    MD5 = {...}
    DEFAULT_JUDGE = ['chatgpt-0125', 'gpt-4-0125']
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with video paths and tampering detection MCQ questions
Outputs	Evaluation results DataFrame with MCQ accuracy scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('MVTamperBench')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment