Principle:Online ml River Streaming Regression Metrics
| Knowledge Sources | Domains | Last Updated |
|---|---|---|
| Machine Learning Statistics | Online_Learning, Evaluation, Regression | 2026-02-08 18:00 GMT |
Overview
Streaming regression metrics are evaluation measures for continuous-valued predictions that are maintained incrementally as prediction-target pairs arrive one at a time. They track running statistics such as cumulative errors and target variance, enabling anytime computation of metrics like MAE, MSE, R-squared, and percentage-based error measures.
Description
Evaluating regression models in the online setting requires metrics that can be updated in constant time and space per observation. Each metric maintains a small set of sufficient statistics and can report its current value at any point during the stream.
The core streaming regression metrics include:
Mean Absolute Error (MAE): The average absolute difference between predicted and actual values. It is robust to outliers and provides an intuitive measure in the same units as the target variable.
Mean Absolute Percentage Error (MAPE): The average absolute error expressed as a percentage of the actual value. It is scale-independent, making it useful for comparing models across different datasets, but can be undefined or misleading when actual values are near zero.
Mean Squared Error (MSE): The average squared difference between predicted and actual values. It penalizes large errors disproportionately, making it sensitive to outliers. The root mean squared error (RMSE) is its square root, bringing it back to the original units.
R-squared (Coefficient of Determination): Measures the proportion of variance in the target that is explained by the model. An R-squared of 1 indicates perfect prediction; 0 indicates the model performs no better than predicting the mean; negative values indicate the model is worse than the mean predictor.
Symmetric Mean Absolute Percentage Error (SMAPE): A symmetrized version of MAPE that addresses the asymmetry problem by normalizing the error by the average of the predicted and actual values, bounding it between 0% and 200%.
Usage
Use streaming regression metrics when:
- You are evaluating an online regression model in a progressive validation loop.
- You need to monitor regression model performance in real time.
- You want to compare multiple regression models on a data stream.
- You need scale-dependent (MAE, MSE) or scale-independent (MAPE, SMAPE, R2) evaluation.
Theoretical Basis
Incremental Sufficient Statistics
All streaming regression metrics can be maintained using running sums:
Initialize: n = 0, sum_ae = 0, sum_se = 0, sum_ape = 0
sum_smape = 0, sum_y = 0, sum_y2 = 0
update(y_true, y_pred):
n += 1
error = y_true - y_pred
sum_ae += |error|
sum_se += error^2
sum_ape += |error| / |y_true| (if y_true != 0)
sum_smape += |error| / (|y_true| + |y_pred|)
sum_y += y_true
sum_y2 += y_true^2
Metric Formulas
MAE = sum_ae / n
MSE = sum_se / n
RMSE = sqrt(MSE)
MAPE = (100 / n) * sum_ape
SMAPE = (200 / n) * sum_smape
R^2 = 1 - sum_se / SS_tot
where SS_tot = sum_y2 - (sum_y)^2 / n (total variance)
Properties
- MAE is the L1 loss average and is minimized by the conditional median.
- MSE is the L2 loss average and is minimized by the conditional mean.
- R-squared is undefined when all target values are identical (zero variance). In the online setting, early R-squared values can be unstable when few observations have been seen.
- SMAPE is bounded in [0, 200%] and avoids the division-by-zero problem of MAPE when predictions are zero.
Each metric requires O(1) storage and O(1) time per update, making them suitable for high-throughput streaming applications.