Principle:Obss Sahi COCO Error Analysis
| Knowledge Sources | |
|---|---|
| Domains | Object_Detection, Evaluation, Error_Analysis |
| Last Updated | 2026-02-08 12:00 GMT |
Overview
A diagnostic technique that decomposes object detection errors into distinct categories (classification, localization, duplicate, background, missed) to identify specific failure modes beyond aggregate AP metrics.
Description
While COCO mAP provides an aggregate measure of detection quality, it does not reveal why a model fails. Error analysis addresses this by categorizing each false positive and false negative into specific error types:
Error Categories (TIDE framework):
- C75: Correct at IoU=0.75 (precise localization)
- C50: Correct at IoU=0.50 but not at 0.75 (moderate localization error)
- Loc: Localization error (IoU between 0.1 and 0.5 with correct class)
- Sim: Confusion with a similar category
- Oth: Confusion with a dissimilar category
- BG: Background false positive (no matching ground truth)
- FN: False negative (missed detection)
By computing precision at each confusion level, error analysis reveals the relative contribution of each error type. This enables targeted model improvements: if most errors are localization, improving anchor design may help; if most are classification errors, better feature learning or data augmentation is needed.
SAHI's implementation performs this analysis per category in parallel (using multiprocessing) and generates precision-recall curve plots for visual inspection.
Usage
Use error analysis after COCO evaluation to diagnose specific model weaknesses. This is particularly valuable when:
- Comparing sliced vs non-sliced inference to understand where slicing helps
- Debugging models that perform well on mAP but poorly in practice
- Identifying which object categories need more training data or better features
Theoretical Basis
The analysis modifies the IoU matching threshold progressively to reveal error sources:
# Pseudocode for TIDE-style error analysis
def analyze_category(cocoEval, category_id):
results = {}
for level in ["C75", "C50", "Loc", "Sim", "Oth", "BG", "FN"]:
# Relax matching criteria progressively
modified_eval = modify_matching(cocoEval, level)
modified_eval.evaluate()
modified_eval.accumulate()
results[level] = extract_precision_recall(modified_eval)
return results
The difference in AP between adjacent confusion levels quantifies the contribution of each error type. For example, if AP jumps significantly from C75 to C50, the model has many predictions with good localization (IoU > 0.5) but not precise localization (IoU < 0.75).
Output includes per-category and overall precision-recall curves exported as PNG plots.