Implementation:Cleanlab Cleanlab Health Summary
| Knowledge Sources | |
|---|---|
| Domains | Machine_Learning, Data_Quality |
| Last Updated | 2026-02-09 19:00 GMT |
Overview
Concrete tool for generating a comprehensive dataset-level label quality diagnostic summary provided by the Cleanlab library.
Description
This function produces a dictionary containing multiple dataset health metrics: an overall label health score (the estimated fraction of correctly labeled examples), the estimated joint distribution matrix, a DataFrame ranking all classes by their label quality, and a DataFrame identifying pairs of classes that are frequently confused (potentially overlapping). When verbose=True, it also prints a human-readable summary to the console. The function accepts either labels and pred_probs (from which it computes the joint) or a pre-computed joint/confident_joint directly.
Usage
Import and use this function as a first step in a dataset quality audit. It provides a comprehensive overview before diving into individual label issues. It is particularly useful for understanding whether labeling problems are concentrated in specific classes or distributed broadly, and for identifying class pairs that may need clearer annotation guidelines.
Code Reference
Source Location
- Repository: cleanlab
- File: cleanlab/dataset.py
- Lines: 361-488
Signature
def health_summary(
labels=None,
pred_probs=None,
*,
asymmetric=False,
class_names=None,
num_examples=None,
joint=None,
confident_joint=None,
multi_label=False,
verbose=True,
) -> dict
Import
from cleanlab.dataset import health_summary
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| labels | Optional[LabelLike] | No | Array of noisy class labels of shape (N,). Required if joint and confident_joint are not provided. |
| pred_probs | Optional[np.ndarray] | No | Out-of-sample predicted probability matrix of shape (N, K). Required if joint and confident_joint are not provided. |
| asymmetric | bool | No | If True, treat class overlap as asymmetric (direction matters). Defaults to False. |
| class_names | Optional[Iterable[str]] | No | Human-readable class names to display instead of integer indices. |
| num_examples | Optional[int] | No | Total number of examples, used when providing a pre-computed joint without labels. |
| joint | Optional[np.ndarray] | No | Pre-computed joint distribution matrix of shape (K, K). If provided, labels and pred_probs are not needed. |
| confident_joint | Optional[np.ndarray] | No | Pre-computed confident joint matrix. Used to compute the joint if joint is not directly provided. |
| multi_label | bool | No | If True, handle multi-label classification. Defaults to False. |
| verbose | bool | No | If True, print a human-readable summary to the console. Defaults to True. |
Outputs
| Name | Type | Description |
|---|---|---|
| result | dict | Dictionary containing: "overall_label_health_score" (float, fraction of correctly labeled examples), "joint" (np.ndarray K x K, estimated joint distribution), "classes_by_label_quality" (pd.DataFrame, classes ranked by quality), "overlapping_classes" (pd.DataFrame, pairs of classes with high confusion). |
Usage Examples
Basic Usage
import numpy as np
from cleanlab.dataset import health_summary
labels = np.array([0, 0, 1, 1, 2, 2, 0, 1, 2, 1])
pred_probs = np.array([
[0.9, 0.05, 0.05],
[0.2, 0.7, 0.1],
[0.1, 0.8, 0.1],
[0.05, 0.1, 0.85],
[0.1, 0.1, 0.8],
[0.05, 0.05, 0.9],
[0.85, 0.1, 0.05],
[0.1, 0.7, 0.2],
[0.0, 0.2, 0.8],
[0.15, 0.75, 0.1],
])
summary = health_summary(labels, pred_probs, verbose=True)
print("Overall health score:", summary["overall_label_health_score"])
print("\nClasses by label quality:")
print(summary["classes_by_label_quality"])
print("\nOverlapping class pairs:")
print(summary["overlapping_classes"])
With Class Names
from cleanlab.dataset import health_summary
summary = health_summary(
labels, pred_probs,
class_names=["cat", "dog", "bird"],
verbose=True,
)
# Output uses class names instead of integer indices
print(summary["classes_by_label_quality"])
From Pre-Computed Confident Joint
from cleanlab.count import compute_confident_joint
from cleanlab.dataset import health_summary
cj = compute_confident_joint(labels, pred_probs)
summary = health_summary(confident_joint=cj, num_examples=len(labels))