Implementation:Open compass VLMEvalKit ChartQAPro Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Chart Question Answering |
Overview
Provides prompt generation and evaluation utilities for the ChartQA Pro benchmark, supporting multiple question types and VQA reasoning modes.
Description
This module implements prompt_context for generating type-aware prompts across five question types (Factoid, Multi Choice, Conversational, Fact Checking, Hypothetical) and three VQA modes (Direct, Chain-of-Thought, Program-of-Thought). It constructs appropriately formatted question prompts with specific instructions for each combination, handling multi-turn conversations and unit formatting requirements. The module also includes answer extraction and evaluation logic for comparing model responses against ground-truth chart answers.
Usage
Called internally by the ChartQA Pro dataset class during prompt construction and evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/chartqapro.py, Lines: L1-355 - Import:
from vlmeval.dataset.utils.chartqapro import prompt_context
Key Functions:
def prompt_context(question, answer, q_type, vqa_type): ...
def evaluate_answer(prediction, ground_truth, q_type): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Question text, answer options, question type (Factoid/Multi Choice/etc.), VQA type (Direct/CoT/PoT) |
| Outputs | Formatted prompt string for model input; evaluation scores comparing prediction against ground truth |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.chartqapro import prompt_context
prompt = prompt_context(question, answer, "Factoid", "Direct")