Implementation:Open compass VLMEvalKit ChartMimic Text Evaluator

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Chart Generation, Text Elements

Overview

Evaluates text element accuracy by comparing text objects (titles, labels, tick labels) extracted from generated and golden matplotlib code in the ChartMimic benchmark.

Description

The `TextEvaluator` class instruments matplotlib code to log all text objects from chart axes. It supports optional axis tick text inclusion via `use_axs` and positional information via `use_position`. When `use_axs` is disabled, it injects code to delete axis tick labels before capturing text objects. The evaluator executes modified scripts via `run_script_safe` and computes precision, recall, and F1 over matched text elements.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/chartmimic/evaluator/text_evaluator.py, Lines: L1-206
• Import: from vlmeval.dataset.utils.chartmimic.evaluator.text_evaluator import TextEvaluator

Key Functions:

class TextEvaluator:
    def __call__(self, generation_code_file, golden_code_file): ...
    def _log_texts(self, code_file): ...
    def _calculate_metrics(self, generation_texts, golden_texts): ...

I/O Contract

Direction	Description
Inputs	Paths to generated and golden Python code files producing matplotlib charts
Outputs	Metrics dict with precision, recall, and F1 scores for text element matching

Usage Examples

from vlmeval.dataset.utils.chartmimic.evaluator.text_evaluator import TextEvaluator

evaluator = TextEvaluator(use_position=False, use_axs=True)
evaluator("generated_chart.py", "golden_chart.py")
print(evaluator.metrics)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction