Implementation:Open compass VLMEvalKit KIE Evaluator
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, OCR, Key Information Extraction |
Overview
Implements Key Information Extraction (KIE) evaluation using tree edit distance for the CCOCR evaluator module.
Description
This module provides the KIE evaluator based on the Donut framework. It includes the flatten function for converting nested dictionaries into flat key-value pairs, update_cost for tree edit distance node comparison, and tree-based evaluation using the ZSS library. The evaluator computes normalized tree edit distances between predicted and ground-truth structured document extractions, supporting nested JSON comparisons with edit distance-based node matching and Counter-based accuracy computation.
Usage
Called internally by the CCOCR dataset class during key information extraction evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py, Lines: L1-385 - Import:
from vlmeval.dataset.utils.ccocr_evaluator.kie_evaluator import flatten, update_cost
Key Functions:
def flatten(data: dict): ...
def update_cost(node1: Node, node2: Node): ...
def construct_tree(data): ...
def compute_tree_edit_distance(pred, gt): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Predicted and ground-truth dictionaries representing extracted key information from documents |
| Outputs | Normalized tree edit distance score indicating extraction accuracy; flattened key-value pair lists |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.ccocr_evaluator.kie_evaluator import flatten, update_cost
flat_data = flatten({"menu": [{"name": ["cake"], "count": ["2"]}]})
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment