Implementation:Open compass VLMEvalKit QSpatial Utils

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Spatial Reasoning, Measurement

Overview

Provides GPT-based answer extraction for the QSpatial benchmark, handling spatial measurement answers in tuple format with numeric values and units.

Description

This module implements `get_gpt4_ICE_for_qspatial` with five in-context examples demonstrating extraction of spatial measurements in (value, unit) tuple format from model responses. Examples cover scenarios including unanswerable questions (returning (0, cm)), distance calculations, height conversions (feet-inches to inches), and direct measurements. The `list_to_dict` helper converts lists to letter-keyed dictionaries for multiple-choice formatting.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/qspatial.py, Lines: L1-123
• Import: from vlmeval.dataset.utils.qspatial import get_gpt4_ICE_for_qspatial, list_to_dict

Key Functions:

def get_gpt4_ICE_for_qspatial(): ...
def list_to_dict(lst): ...

I/O Contract

Direction	Description
Inputs	Model response strings containing spatial measurements with values and units
Outputs	List of in-context example strings; letter-keyed dictionaries from lists

Usage Examples

from vlmeval.dataset.utils.qspatial import get_gpt4_ICE_for_qspatial

examples = get_gpt4_ICE_for_qspatial()

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction