Implementation:Online ml River Ensemble EWARegressor

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Ensemble_Methods, Hedging
Last Updated	2026-02-08 16:00 GMT

Overview

Exponentially Weighted Average (EWA) regressor hedges multiple models by weighting their predictions based on exponentially discounted loss history.

Description

EWA implements the multiplicative weight update method for online learning. Each model in the ensemble maintains a weight that is exponentially decreased based on its loss. The algorithm follows a learn-then-predict cycle: for each instance, models make predictions, weights are updated using exp(-learning_rate * loss), predictions are aggregated using current weights, and weights are normalized to sum to 1. This creates a hedging strategy that quickly reduces influence of poorly performing models while maintaining diversity.

Usage

Use EWA when you have multiple models or optimization strategies and want automatic selection without prior knowledge of which performs best. It's particularly effective when different models excel in different regions of the input space. The learning_rate parameter controls adaptation speed - higher values respond faster to performance changes but may be more volatile. EWA often outperforms individual models by intelligently combining their strengths.

Code Reference

Source Location

Repository: Online_ml_River
File: river/ensemble/ewa.py

Signature

class EWARegressor(base.Ensemble, base.Regressor):
    def __init__(
        self,
        models: list[base.Regressor],
        loss: optim.losses.RegressionLoss | None = None,
        learning_rate=0.5,
    ):
        super().__init__(models)
        self.loss = optim.losses.Squared() if loss is None else loss
        self.learning_rate = learning_rate
        self.weights = [1.0] * len(models)

Import

from river import ensemble

I/O Contract

Parameters

Parameter	Type	Default	Description
models	list[Regressor]	required	List of regressors to hedge
loss	RegressionLoss or None	Squared()	Loss function for weight updates
learning_rate	float	0.5	Weight update rate (higher = faster adaptation)

Attributes

Attribute	Type	Description
weights	list[float]	Current normalized weights per model

Input/Output

Method	Input	Output
learn_one	x: dict, y: float	None
learn_predict_one	x: dict, y: float	float
predict_one	x: dict	float

Usage Examples

from river import datasets
from river import ensemble
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing

# First, compare individual optimizers
optimizers = [
    optim.SGD(0.01),
    optim.RMSProp(),
    optim.AdaGrad()
]

for optimizer in optimizers:
    dataset = datasets.TrumpApproval()
    metric = metrics.MAE()
    model = (
        preprocessing.StandardScaler() |
        linear_model.LinearRegression(
            optimizer=optimizer,
            intercept_lr=.1
        )
    )

    print(optimizer, evaluate.progressive_val_score(dataset, model, metric))
# SGD MAE: 0.558735
# RMSProp MAE: 0.522449
# AdaGrad MAE: 0.477289

# Now use EWA to hedge them
dataset = datasets.TrumpApproval()
metric = metrics.MAE()
hedge = (
    preprocessing.StandardScaler() |
    ensemble.EWARegressor(
        [
            linear_model.LinearRegression(optimizer=o, intercept_lr=.1)
            for o in optimizers
        ],
        learning_rate=0.005
    )
)

evaluate.progressive_val_score(dataset, hedge, metric)
# MAE: 0.496298
# EWA beats all individual optimizers!

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment