Implementation:Cleanlab Cleanlab Multilabel Health Summary

Knowledge Sources	Cleanlab
Domains	Machine Learning, Data Quality, Multi-Label Classification
Last Updated	2026-02-09 00:00 GMT

Overview

The multilabel_health_summary function prints and returns a comprehensive health report for a multi-label classification dataset, aggregating per-class quality scores, common mislabeling patterns, and an overall label health score.

Description

This module in cleanlab.multilabel_classification.dataset provides four public functions for dataset-level diagnostics of multi-label classification labels:

• multilabel_health_summary: The top-level entry point that generates a full diagnostic report combining class quality rankings, common issue patterns, and an overall health score. It prints formatted output and returns a dictionary containing all results.
• common_multilabel_issues: Summarizes which classes are most often mislabeled by counting, for each class independently, the number of examples where the label is True but should be False (and vice versa). Returns a DataFrame sorted by issue probability.
• rank_classes_by_multilabel_quality: Computes per-class quality metrics including label noise rate, inverse label noise rate, and a label quality score (1 minus noise rate). Returns a DataFrame sorted by quality score in ascending order so the most problematic classes appear first.
• overall_multilabel_health_score: Returns a single float between 0 and 1 representing the fraction of examples that appear correctly labeled across all classes.

The module decomposes the multi-label problem into independent per-class binary classification subproblems, delegating the actual issue detection to find_label_issues and find_multilabel_issues_per_class from the filter module.

Usage

Import this module when you need to assess the overall quality of a multi-label classification dataset. Use multilabel_health_summary for a complete diagnostic report, or call individual functions when you need specific metrics such as per-class quality rankings or a single overall health score.

Code Reference

Source Location

• Repository: Cleanlab
• File: cleanlab/multilabel_classification/dataset.py
• Lines: 1-327

Signature

def multilabel_health_summary(
    labels=None,
    pred_probs=None,
    *,
    class_names=None,
    num_examples=None,
    confident_joint=None,
    verbose=True,
) -> Dict

def common_multilabel_issues(
    labels=list,
    pred_probs=None,
    *,
    class_names=None,
    confident_joint=None,
) -> pd.DataFrame

def rank_classes_by_multilabel_quality(
    labels=None,
    pred_probs=None,
    *,
    class_names=None,
    joint=None,
    confident_joint=None,
) -> pd.DataFrame

def overall_multilabel_health_score(
    labels=None,
    pred_probs=None,
    *,
    confident_joint=None,
) -> float

Import

from cleanlab.multilabel_classification.dataset import multilabel_health_summary
from cleanlab.multilabel_classification.dataset import common_multilabel_issues
from cleanlab.multilabel_classification.dataset import rank_classes_by_multilabel_quality
from cleanlab.multilabel_classification.dataset import overall_multilabel_health_score

I/O Contract

Inputs (multilabel_health_summary)

Name	Type	Required	Description
labels	List[List[int]]	Yes	List of noisy labels where each element is a list of class indices the example belongs to.
pred_probs	np.ndarray (N, K)	Yes	Model-predicted class probabilities. Columns need not sum to 1.
class_names	Iterable[str]	No	Optional list of string class names matching label index order.
num_examples	int	No	Number of examples in the dataset. Inferred from labels if not provided.
confident_joint	np.ndarray (K, 2, 2)	No	One-vs-rest formatted confident joint. Computed automatically if not provided.
verbose	bool	No	If True (default), prints the formatted health summary to stdout.

Outputs (multilabel_health_summary)

Name	Type	Description
summary	Dict	Dictionary with keys: "overall_multilabel_health_score" (float), "classes_by_multilabel_quality" (pd.DataFrame), and "common_multilabel_issues" (pd.DataFrame).

Outputs (common_multilabel_issues)

Name	Type	Description
return	pd.DataFrame	DataFrame with columns: Class Index, In Given Label, In Suggested Label, Num Examples, Issue Probability. Optionally includes Class Name. Sorted by Issue Probability descending.

Outputs (rank_classes_by_multilabel_quality)

Name	Type	Description
return	pd.DataFrame	DataFrame with columns: Class Index, Label Issues, Inverse Label Issues, Label Noise, Inverse Label Noise, Label Quality Score. Optionally includes Class Name. Sorted by Label Quality Score ascending.

Outputs (overall_multilabel_health_score)

Name	Type	Description
return	float	Score between 0 and 1, where 1 means all labels appear correct and 0.5 means roughly half have issues.

Usage Examples

Basic Usage

from cleanlab.multilabel_classification.dataset import multilabel_health_summary

# labels: each example lists which classes it belongs to
labels = [[0, 1], [1], [0, 2], [2], [0, 1, 2]]

# pred_probs: model-predicted probabilities, shape (N, K)
import numpy as np
pred_probs = np.array([
    [0.9, 0.8, 0.1],
    [0.2, 0.9, 0.1],
    [0.8, 0.1, 0.7],
    [0.1, 0.2, 0.9],
    [0.7, 0.8, 0.6],
])

# Generate the full health summary
summary = multilabel_health_summary(
    labels=labels,
    pred_probs=pred_probs,
    class_names=["cat", "dog", "bird"],
)

# Access individual results
print("Health score:", summary["overall_multilabel_health_score"])
print(summary["classes_by_multilabel_quality"])
print(summary["common_multilabel_issues"])

Get Overall Health Score Only

from cleanlab.multilabel_classification.dataset import overall_multilabel_health_score

score = overall_multilabel_health_score(labels=labels, pred_probs=pred_probs)
print(f"Dataset health: {score:.2%}")

Related Pages

• Principle:Cleanlab_Cleanlab_Multilabel_Dataset_Health_Analysis