Implementation:Open compass VLMEvalKit TEDS Metric

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, OCR, Table Structure Recognition

Overview

Implements the Tree Edit Distance-based Similarity (TEDS) metric for evaluating table structure recognition accuracy in the OCRBench v2 benchmark.

Description

This module provides the TEDS class and supporting utilities for computing tree edit distance between predicted and ground-truth HTML tables. It converts HTML table representations into tree structures using TableTree and CustomConfig classes built on the APTED library, then computes normalized edit distances. The module also includes functions for converting markdown tables to HTML, computing F1 scores, and evaluating document parsing outputs.

Usage

Called internally by the OCRBench v2 dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py, Lines: L1-930
• Import: from vlmeval.dataset.utils.Ocrbench_v2.TEDS_metric import TEDS, convert_markdown_table_to_html

Key Functions:

class TableTree(Tree):
    def __init__(self, tag, colspan=None, rowspan=None, content=None, *children): ...

class CustomConfig(Config):
    def normalized_distance(self, *sequences): ...

class TEDS:
    def __init__(self, n_jobs=32): ...
    def evaluate(self, pred_html, gt_html): ...

def convert_markdown_table_to_html(markdown_table): ...
def compute_f1_score(prediction, ground_truth): ...
def doc_parsing_evaluation(pred, gt): ...

I/O Contract

Direction	Description
Inputs	Predicted and ground-truth HTML table strings for TEDS computation; markdown table strings for conversion
Outputs	Normalized similarity score between 0.0 and 1.0 (1.0 = identical structure); F1 scores for document parsing

Usage Examples

# Internal usage example
from vlmeval.dataset.utils.Ocrbench_v2.TEDS_metric import TEDS, convert_markdown_table_to_html
teds = TEDS(n_jobs=32)
score = teds.evaluate(pred_html, gt_html)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction