Principle:Cleanlab Cleanlab Regression Label Quality Scoring
| Knowledge Sources | |
|---|---|
| Domains | Data Quality, Machine Learning, Regression |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
Regression label quality scoring quantifies how likely each continuous target value in a regression dataset is to be erroneous, using residual analysis and outlier detection techniques.
Description
In regression tasks, label errors manifest as Y-values that deviate significantly from what a well-trained model would predict. Unlike classification where a label is simply wrong or right, regression label errors exist on a spectrum: a label can be slightly off or drastically incorrect. Label quality scoring for regression assigns a continuous score between 0 and 1 to each example, where lower scores indicate greater likelihood of the label being erroneous.
Two complementary approaches address this problem:
Residual-based scoring uses the simple intuition that the larger the gap between a model's prediction and the given label, the more likely the label is wrong. By applying an exponential decay function to absolute residuals, scores naturally fall in the [0, 1] range, with perfectly predicted examples receiving scores near 1.
Outlier-based scoring (OUTRE) takes a more sophisticated approach by considering the context of each example relative to its neighbors. Rather than looking at residuals in isolation, it constructs a 2D feature space from normalized labels and scaled residuals, then identifies points that are outliers in this joint space. This captures cases where an example has an unusually large residual relative to examples with similar label values, which can reveal errors that pure residual magnitude would miss.
Usage
Regression label quality scoring is the right choice when:
- You have a regression model and want to audit the quality of your training labels.
- You need to prioritize which examples to review for potential labeling errors.
- You want a lightweight scoring approach without retraining the model (as opposed to the full CleanLearning pipeline).
- You have out-of-sample predictions (e.g., from cross-validation) for every example in your dataset.
The OUTRE method is generally preferred because it accounts for local context, making it more robust to heteroscedastic noise (where residual variance differs across the label range). The residual method is simpler and may be adequate when label noise is relatively uniform.
Theoretical Basis
Residual Scoring
The residual method computes scores as:
score(i) = exp(-|prediction_i - label_i|)
This maps absolute residuals to [0, 1] via exponential decay. A residual of 0 yields a score of 1 (perfect), while increasingly large residuals yield scores approaching 0.
OUTRE Scoring
The OUTRE (OUTlier-in-REsidual-space) method operates through the following logical steps:
- Normalization: Labels and residuals are independently standardized to zero mean and unit variance, removing scale dependencies.
- Residual scaling: Residuals are multiplied by a scale factor (default 5) to emphasize residual deviations relative to label variation, since large residuals are the primary signal of label error.
- Feature construction: Normalized labels and scaled residuals are combined into a 2D feature matrix, creating a joint representation of "where an example sits in label space" and "how wrong its label appears."
- Neighbor-based outlier detection: A k-nearest-neighbors graph (with k = 50% of the data) is constructed, and the OutOfDistribution scorer evaluates how anomalous each point is in this 2D space.
- Score output: The outlier scores are returned directly as label quality scores, with lower scores indicating more anomalous (likely mislabeled) examples.
The key insight is that outlier detection in the label-residual space captures contextual label errors: examples whose residuals are unusually large compared to their neighbors in label space, not just large in absolute terms.