Implementation:Open compass VLMEvalKit Spotting Eval Script

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, OCR, Text Spotting, End-to-End Recognition

Overview

Implements end-to-end text recognition evaluation using Intersection over Union (IoU) criteria for the OCRBench v2 text spotting benchmark.

Description

This module provides the main evaluation script for end-to-end text recognition, computing precision, recall, and F-measure using IoU-based matching between detected and ground-truth text regions. Key functions include default_evaluation_params for configuring IoU thresholds and detection parameters, validate_data for input validation, and the main evaluation logic supporting both word spotting and direct matching modes. It also supports Average Precision calculation when confidence scores are provided.

Usage

Called internally by the OCRBench v2 spotting evaluation pipeline during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/script.py, Lines: L1-451
• Import: from vlmeval.dataset.utils.Ocrbench_v2.spotting_eval.script import default_evaluation_params, validate_data

Key Functions:

def evaluation_imports(): ...
def default_evaluation_params(): ...
def validate_data(gtFilePath, submFilePath, evaluationParams): ...
def evaluate_method(gtFilePath, submFilePath, evaluationParams): ...

I/O Contract

Direction	Description
Inputs	Ground-truth and submission ZIP file paths containing text detection results with bounding boxes and transcriptions
Outputs	Dictionary containing precision, recall, F-measure, and optionally Average Precision scores

Usage Examples

# Internal usage example
from vlmeval.dataset.utils.Ocrbench_v2.spotting_eval.script import default_evaluation_params, evaluate_method
params = default_evaluation_params()
results = evaluate_method(gt_path, subm_path, params)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction