Implementation:Open compass VLMEvalKit IoUScore Metric

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, OCR, Bounding Box

Overview

Provides IoU (Intersection over Union) scoring for bounding box predictions combined with VQA content evaluation in OCRBench v2.

Description

This module implements bounding box IoU calculation and a combined VQA-with-position evaluation metric for OCRBench v2. The `calculate_iou` function computes the overlap ratio between predicted and ground truth bounding boxes. The `vqa_with_position_evaluation` function combines content accuracy (via VQA evaluation) and spatial accuracy (via IoU) with equal weighting (0.5 each). Helper functions `extract_coordinates` parse coordinate patterns from text responses.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/Ocrbench_v2/IoUscore_metric.py, Lines: L1-87
• Import: from vlmeval.dataset.utils.Ocrbench_v2.IoUscore_metric import calculate_iou, vqa_with_position_evaluation

Key Functions:

def calculate_iou(box1, box2): ...
def vqa_with_position_evaluation(predict, img_metas): ...
def extract_coordinates(text): ...

I/O Contract

Direction	Description
Inputs	Two bounding boxes as coordinate lists [x1, y1, x2, y2]; or a predict dict with "answer" and "bbox" keys plus img_metas with ground truth
Outputs	Float IoU score (0-1) for bounding box overlap; combined float score (0-1) for VQA with position evaluation

Usage Examples

from vlmeval.dataset.utils.Ocrbench_v2.IoUscore_metric import calculate_iou, vqa_with_position_evaluation

iou = calculate_iou([10, 20, 100, 200], [15, 25, 105, 210])
score = vqa_with_position_evaluation(
    {"answer": "hello", "bbox": "[10, 20, 100, 200]"},
    {"answers": ["hello"], "bbox": [10, 20, 100, 200]}
)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction