Implementation:Open compass VLMEvalKit CCOCR OCR Evaluator
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, OCR, Text Recognition |
Overview
Implements OCR-specific evaluation with token normalization, precision/recall/F1 computation, and text tokenization for the CCOCR benchmark.
Description
This module provides OCR evaluation functions including `token_normalize` for case and alphanumeric normalization, `text_normalize_and_tokenize` for preprocessing text into tokens, and `evaluate_single_sample` for computing token-level overlap between predictions and ground truth using counter-based matching. The `calculate_metrics` function computes both macro-averaged and micro-averaged precision, recall, and F1 scores across all samples.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/ccocr_evaluator/ocr_evaluator.py, Lines: L1-106 - Import:
from vlmeval.dataset.utils.ccocr_evaluator.ocr_evaluator import calculate_metrics, token_normalize
Key Functions:
def token_normalize(token_text, is_lower=False, is_alphanum_only=False): ...
def text_normalize_and_tokenize(text, ...): ...
def evaluate_single_sample(gts, preds): ...
def calculate_metrics(response_info, gt_info, is_verbose=False): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Response and ground truth dictionaries mapping filenames to token lists |
| Outputs | Dictionary with macro/micro precision, recall, and F1 scores |
Usage Examples
from vlmeval.dataset.utils.ccocr_evaluator.ocr_evaluator import calculate_metrics
metrics = calculate_metrics(response_info, gt_info)
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment