Implementation:Online ml River Metrics ROCAUC

Knowledge Sources	River River Docs
Domains	Online_Learning Evaluation Classification
Last Updated	2026-02-08 16:00 GMT

Overview

Concrete tool for incrementally approximating the Area Under the Receiver Operating Characteristic Curve (ROC AUC) for streaming binary classification, using discretized thresholds and trapezoidal integration.

Description

The metrics.ROCAUC class computes an approximation of the ROC AUC by maintaining an array of n_thresholds confusion matrices, one per discretized probability threshold. Each confusion matrix independently tracks how the classifier performs when using its corresponding threshold as the decision boundary.

When update(y_true, y_pred) is called, the method extracts the predicted probability of the positive class (either from a dictionary or a scalar) and updates each confusion matrix based on whether the predicted probability exceeds its threshold.

When get() is called, the method computes the True Positive Rate (TPR) and False Positive Rate (FPR) at each threshold from the confusion matrices, then integrates the resulting ROC curve using scipy.integrate.trapezoid.

Key implementation details:

Thresholds are evenly spaced in [0, 1] with small epsilon offsets at the boundaries to ensure proper edge handling.
The class inherits from metrics.base.BinaryMetric and sets requires_labels = False, meaning it expects probability predictions rather than hard labels.
It works with anomaly detectors and anomaly filters in addition to classifiers.
The revert method supports removing previously seen observations.

Usage

Import this class when you need to:

Evaluate a binary classifier's ranking quality in a streaming setting.
Use a threshold-independent metric that is robust to class imbalance.
Compare models based on their ability to separate classes rather than at a fixed threshold.
Pass a metric to evaluate.progressive_val_score that leverages probability predictions.

Code Reference

Source Location

File	Lines
`river/metrics/roc_auc.py`	L10-L103

External Dependencies

Package	Function	Usage
`scipy`	`scipy.integrate.trapezoid`	Trapezoidal integration for AUC computation.

Signature

class ROCAUC(metrics.base.BinaryMetric):
    def __init__(self, n_thresholds=10, pos_val=True)

    def update(self, y_true, y_pred, w=1.0)
    def revert(self, y_true, y_pred, w=1.0)
    def get(self) -> float

Import

from river import metrics

metric = metrics.ROCAUC()

I/O Contract

Inputs

Parameter	Type	Default	Description
`n_thresholds`	`int`	`10`	Number of discretized thresholds for approximating the ROC curve. Higher values yield more accurate results at the cost of more memory and computation.
`pos_val`	any hashable	`True`	The value to treat as the positive class.
`y_true` (to `update`)	any hashable	(required)	The true class label.
`y_pred` (to `update`)	float	(required)	Either a dictionary mapping class labels to probabilities (e.g., `{False: 0.3, True: 0.7}`) or a scalar probability of the positive class.
`w` (to `update`)	`float`	`1.0`	Observation weight.

Outputs

Method	Return Type	Description
`get()`	`float`	AUC value in `[0.0, 1.0]`, computed as `-trapezoid(x=fprs, y=tprs)`. Returns `0.0` if no observations have been processed.
`__repr__()`	`str`	Formatted string such as `"ROCAUC: 95.07%"`.

Usage Examples

Basic incremental ROCAUC:

from river import metrics

y_true = [0, 0, 1, 1]
y_pred = [0.1, 0.4, 0.35, 0.8]

metric = metrics.ROCAUC()
for yt, yp in zip(y_true, y_pred):
    metric.update(yt, yp)

print(metric)
# ROCAUC: 87.50%

With higher precision (more thresholds):

from river import metrics

metric = metrics.ROCAUC(n_thresholds=20)

y_true = [0, 0, 1, 1]
y_pred = [0.1, 0.4, 0.35, 0.8]

for yt, yp in zip(y_true, y_pred):
    metric.update(yt, yp)

print(metric)
# ROCAUC: 75.00%

With progressive validation:

from river import datasets, evaluate, linear_model, metrics, preprocessing

dataset = datasets.Phishing()
model = preprocessing.StandardScaler() | linear_model.LogisticRegression()
metric = metrics.ROCAUC()

evaluate.progressive_val_score(dataset, model, metric)
# ROCAUC: 95.07%

With dictionary predictions:

from river import metrics

metric = metrics.ROCAUC()

# predict_proba_one returns {False: 0.3, True: 0.7}
metric.update(y_true=True, y_pred={False: 0.3, True: 0.7})
metric.update(y_true=False, y_pred={False: 0.8, True: 0.2})

print(metric.get())
# 1.0 (perfect separation)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment