Implementation:Cleanlab Cleanlab Spurious Correlation Detection

Knowledge Sources	Cleanlab
Domains	Data Quality, Computer Vision, Statistical Analysis
Last Updated	2026-02-09 00:00 GMT

Overview

SpuriousCorrelations detects spurious correlations between individual dataset properties and class labels by measuring how predictive each property is of the label compared to a baseline, using cross-validated classification.

Description

The SpuriousCorrelations dataclass takes a DataFrame of property scores (e.g., image quality metrics from CleanVision such as brightness, blurriness, or contrast) and an array of integer class labels. For each property, it trains a GaussianNB classifier using 5-fold cross-validation on that single feature to predict the labels, then computes a relative room for improvement score that quantifies how much more predictive the property is than the baseline (most frequent class prediction).

The score is defined as (1 - mean_accuracy) / (1 - baseline_accuracy), yielding a value between 0 and 1 where:

• Lower scores indicate the property is highly predictive of the label (stronger spurious correlation).
• Higher scores (closer to 1.0) indicate the property is no more predictive than random guessing by majority class.

The module also provides the standalone relative_room_for_improvement() function for computing this metric independently.

Usage

Use SpuriousCorrelations when auditing datasets (particularly image datasets) for properties that are suspiciously correlated with class labels. This is triggered by the Datalab imagelab adapter when image-specific issue checks are enabled, and is useful for detecting dataset bias or shortcut features that models might exploit instead of learning genuine patterns.

Code Reference

Source Location

• Repository: Cleanlab
• File: cleanlab/datalab/internal/spurious_correlation.py
• Lines: 1-113

Signature

@dataclass
class SpuriousCorrelations:
    data: pd.DataFrame
    labels: Union[np.ndarray, list]
    properties_of_interest: Optional[List[str]] = None

    def __post_init__(self): ...
    def calculate_correlations(self) -> pd.DataFrame: ...
    def _get_baseline(self) -> float: ...
    def calculate_spurious_correlation(
        self, property_of_interest, baseline_accuracy: float
    ) -> float: ...


def _train_and_eval(X, y, cv=5) -> float: ...

def relative_room_for_improvement(
    baseline_accuracy: float, mean_accuracy: float, eps: float = 1e-8
) -> float: ...

Import

from cleanlab.datalab.internal.spurious_correlation import SpuriousCorrelations
from cleanlab.datalab.internal.spurious_correlation import relative_room_for_improvement

I/O Contract

Inputs (Constructor)

Name	Type	Required	Description
data	pd.DataFrame	Yes	DataFrame of property scores for the dataset, where each column represents a different property (e.g., brightness, blurriness).
labels	Union[np.ndarray, list]	Yes	Integer class labels for each example. Must have the same length as the number of rows in data.
properties_of_interest	Optional[List[str]]	No	List of column names from data to analyze. Defaults to all columns if not specified.

Outputs (calculate_correlations)

Name	Type	Description
return	pd.DataFrame	DataFrame with columns property (property name) and score (relative room for improvement, where lower values indicate stronger spurious correlation).

Inputs/Outputs (relative_room_for_improvement)

Name	Type	Required	Description
baseline_accuracy	float	Yes	Accuracy of the baseline model (most frequent class). Must be between 0 and 1.
mean_accuracy	float	Yes	Accuracy of the trial model (cross-validated GaussianNB). Must be between 0 and 1.
eps	float	No	Small constant to avoid division by zero when baseline accuracy is 1.0. Default is 1e-8.
return	float	--	The relative room for improvement score, bounded between 0 and 1.

Internal Methods

_get_baseline

Computes the baseline accuracy as the fraction of examples belonging to the most frequent class. This represents the accuracy achievable by always predicting the majority class.

_train_and_eval

Module-level function that trains a GaussianNB classifier with 5-fold cross-validation on a single feature and returns the mean cross-validated accuracy.

Usage Examples

Basic Usage

import numpy as np
import pandas as pd
from cleanlab.datalab.internal.spurious_correlation import SpuriousCorrelations

# Property scores for each image (e.g., from CleanVision)
data = pd.DataFrame({
    "brightness": [0.2, 0.3, 0.8, 0.9, 0.1, 0.4, 0.7, 0.85],
    "blurriness": [0.5, 0.6, 0.4, 0.3, 0.55, 0.65, 0.45, 0.35],
})

# Class labels
labels = np.array([0, 0, 1, 1, 0, 0, 1, 1])

sc = SpuriousCorrelations(data=data, labels=labels)
correlation_scores = sc.calculate_correlations()
print(correlation_scores)
# Properties with low scores are suspiciously correlated with labels

Using relative_room_for_improvement Directly

from cleanlab.datalab.internal.spurious_correlation import relative_room_for_improvement

# Baseline: 60% accuracy (majority class)
# Model: 90% accuracy on a single feature
score = relative_room_for_improvement(baseline_accuracy=0.6, mean_accuracy=0.9)
print(score)  # 0.25 -- low score indicates strong spurious correlation

Related Pages

• Principle:Cleanlab_Cleanlab_Spurious_Correlation_Analysis