Implementation:Open compass VLMEvalKit CreationMMBenchDataset

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Creative Generation

Overview

Benchmark dataset implementation for Creation MMBench creative visual question answering evaluation in VLMEvalKit.

Description

CreationMMBenchDataset inherits from ImageBaseDataset and implements the Creation MMBench benchmark for evaluating creative generation and understanding capabilities of multimodal models. The TYPE field is set to 'CreationVQA'. It supports single and multi-image inputs with custom image dumping logic and handles both base64-encoded and path-referenced images.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/creation.py, Lines: L1-741
Import: from vlmeval.dataset.creation import CreationMMBenchDataset

Signature:

class CreationMMBenchDataset(ImageBaseDataset):
    TYPE = 'CreationVQA'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with images and creative generation tasks
Outputs	Evaluation results DataFrame with creative quality scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('Creation_MMBench')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment