Implementation:Open compass VLMEvalKit CCOCR Common
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, OCR, Document Parsing |
Overview
Provides common utility functions and a base metric class for the CCOCR (Cross-lingual Chinese OCR) evaluator suite, including response text extraction and metric computation.
Description
This module defines `pick_response_text` for extracting text from various model API response formats (GPT, Claude, Gemini, Qwen, local models) and `load_response_from_dir` for batch loading response files. It also provides `BaseMetric` as a foundation class and core evaluation logic including `evaluate_single_sample` for token-level matching, `calculate_metrics` for computing macro/micro precision, recall, and F1 scores, and `text_normalize_and_tokenize` for text preprocessing.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/ccocr_evaluator/common.py, Lines: L1-222 - Import:
from vlmeval.dataset.utils.ccocr_evaluator.common import BaseMetric, calculate_metrics
Key Functions:
def pick_response_text(json_path): ...
def load_response_from_dir(res_dir): ...
def calculate_metrics(response_info, gt_info, is_verbose=False): ...
def text_normalize_and_tokenize(text, ...): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | JSON response files or response/ground-truth dictionaries mapping file names to token lists |
| Outputs | Dictionary with macro/micro precision, recall, F1 scores |
Usage Examples
from vlmeval.dataset.utils.ccocr_evaluator.common import calculate_metrics
metrics = calculate_metrics(response_info, gt_info)
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment