Implementation:Open compass VLMEvalKit MMLongBench

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Long Document Understanding

Overview

Benchmark dataset implementation for MMLongBench long document understanding evaluation in VLMEvalKit.

Description

MMLongBench inherits from ImageBaseDataset and implements the MMLongBench_DOC benchmark for evaluating multimodal models on long document understanding tasks. The TYPE field is set to 'VQA'. It includes a SUPPORTED_MODELS mapping defining per-model page handling configurations and handles multi-page document processing with configurable page limits.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/mmlongbench.py, Lines: L1-584
Import: from vlmeval.dataset.mmlongbench import MMLongBench

Signature:

class MMLongBench(ImageBaseDataset):
    TYPE = 'VQA'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    SUPPORTED_MODELS = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with multi-page document images and questions
Outputs	Evaluation results DataFrame with document understanding scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('MMLongBench_DOC')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment