Implementation:Open compass VLMEvalKit ChartMimic

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Chart Understanding, Code Generation

Overview

Benchmark dataset implementation for ChartMimic chart reproduction and understanding evaluation in VLMEvalKit.

Description

ChartMimic inherits from ImageBaseDataset and implements the ChartMimic benchmark for evaluating models on chart understanding and code generation tasks. The TYPE field is set to 'VQA'. It supports multiple versions (v1, v2) and modes (customized, direct) with varying dataset sizes (600, 1800), and includes comprehensive evaluation of text, legend, layout, grid, color, and chart type accuracy.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/chartmimic.py, Lines: L1-785
Import: from vlmeval.dataset.chartmimic import ChartMimic

Signature:

class ChartMimic(ImageBaseDataset):
    TYPE = "VQA"
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with chart images and reproduction/understanding tasks
Outputs	Evaluation results DataFrame with multi-dimensional scores

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('ChartMimic_v1_customized')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment