Implementation:Open compass VLMEvalKit ImageMCQDataset Build Prompt

Field	Value
source	VLMEvalKit
domain	Vision, NLP, Evaluation

Overview

Concrete tool for constructing standardized MCQ prompts from benchmark data rows with options formatting provided by VLMEvalKit.

Description

ImageMCQDataset.build_prompt() in vlmeval/dataset/image_mcq.py (L199-234) constructs a multimodal message from a data row. It: (1) dumps/decodes images from the row, (2) extracts options from columns A-Z (skipping NaN), (3) formats as "Options:\nA. text\nB. text\n...", (4) adds optional hint, (5) combines into "Hint: ...\nQuestion: ...\nOptions:\n...\nPlease select the correct answer from the options above.", (6) returns a message list of image dicts + text dict.

Usage

Called by the inference pipeline for all MCQ datasets unless the model provides a custom prompt via use_custom_prompt().

Code Reference

• Source: vlmeval/dataset/image_mcq.py, Lines: L199-234
• Signature:

def build_prompt(self, line: Union[int, pd.Series]) -> List[Dict[str, str]]:
    """
    Builds MCQ prompt from a data row.
    Args:
        line: Data row (pd.Series) or integer index.
    Returns:
        List of message dicts: [{'type': 'image', 'value': path}, ..., {'type': 'text', 'value': prompt}]
    """

• Import: (method on ImageMCQDataset) from vlmeval.dataset import ImageMCQDataset

I/O Contract

Direction	Name	Type	Description
Input	line	pd.Series or int	Data row with columns: question, image, A, B, C, D (optional), hint (optional)
Output	(message list)	List[Dict]	Message list with image(s) and formatted text prompt

Usage Examples

from vlmeval.dataset import build_dataset

dataset = build_dataset("MMBench_DEV_EN_V11")
# Build prompt for first sample
prompt = dataset.build_prompt(dataset.data.iloc[0])
# prompt is like:
# [
#     {'type': 'image', 'value': '/path/to/image.jpg'},
#     {'type': 'text', 'value': 'Question: What is shown?\nOptions:\nA. Cat\nB. Dog\nC. Bird\nD. Fish\nPlease select the correct answer from the options above.\n'}
# ]
print(prompt[-1]['value'])

Related Pages

• Principle:Open_compass_VLMEvalKit_MCQ_Prompt_Construction