Implementation:Online ml River Metrics RollingROCAUC
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Evaluation_Metrics |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
Rolling window ROC AUC metric computing exact area under ROC curve over the last S instances.
Description
RollingROCAUC calculates the Receiving Operating Characteristic Area Under the Curve using only instances within a sliding window of size S. Unlike standard ROCAUC which approximates over the entire stream, RollingROCAUC computes the exact value for the last S instances using an efficient tree-based algorithm. As new instances arrive, old ones beyond the window are removed, providing a metric that reflects current model performance rather than historical average.
Usage
Use RollingROCAUC when you need to evaluate binary classifier performance over recent predictions rather than the entire stream history. This is particularly valuable for detecting concept drift or monitoring model performance over time, as the metric can reveal degradation that might be hidden by cumulative metrics. The window size controls the tradeoff between statistical stability (larger windows) and sensitivity to recent changes (smaller windows).
Code Reference
Source Location
- Repository: Online_ml_River
- File: river/metrics/rolling_roc_auc.py
Signature
class RollingROCAUC(metrics.base.BinaryMetric):
def __init__(self, window_size=1000, pos_val=True):
pass
Import
from river import metrics
I/O Contract
| Method | Parameters | Returns | Description |
|---|---|---|---|
| update | y_true, y_pred (probability) | None | Updates with true label and predicted probability |
| get | - | float | Returns ROC AUC over the window (0.0 to 1.0) |
Note: requires_labels returns False (requires probabilities). Does not support sample weights (works_with_weights returns False).
Usage Examples
from river import metrics
y_true = [ 0, 1, 0, 1, 0, 1, 0, 0, 1, 1]
y_pred = [.3, .5, .5, .7, .1, .3, .1, .4, .35, .8]
metric = metrics.RollingROCAUC(window_size=4)
for yt, yp in zip(y_true, y_pred):
metric.update(yt, yp)
print(metric)
# RollingROCAUC: 75.00%
# The window size of 4 means only the last 4 instances
# affect the current ROC AUC score
# Compare with larger window for more stability
metric_large = metrics.RollingROCAUC(window_size=10)
for yt, yp in zip(y_true, y_pred):
metric_large.update(yt, yp)
print(metric_large)
# Larger window provides more stable estimates
# Use case: Detecting concept drift
# As model performance changes over time, rolling window
# will reflect this faster than cumulative metrics
import random
random.seed(42)
# Simulate good then poor performance
y_true_drift = [random.randint(0, 1) for _ in range(100)]
y_pred_good = [0.8 if y == 1 else 0.2 for y in y_true_drift[:50]]
y_pred_poor = [0.4 if y == 1 else 0.6 for y in y_true_drift[50:]]
y_pred_drift = y_pred_good + y_pred_poor
metric_drift = metrics.RollingROCAUC(window_size=20)
for i, (yt, yp) in enumerate(zip(y_true_drift, y_pred_drift)):
metric_drift.update(yt, yp)
if i % 20 == 19:
print(f"Sample {i+1}: ROC AUC = {metric_drift.get():.2%}")
# ROC AUC will drop when poor predictions enter the window