Implementation:Online ml River Optim RMSProp

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Optimization
Last Updated	2026-02-08 16:00 GMT

Overview

RMSProp (Root Mean Square Propagation) is an adaptive learning rate optimizer that divides the learning rate by an exponentially decaying average of squared gradients.

Description

RMSProp addresses the diminishing learning rates problem of AdaGrad by using an exponentially decaying average of past squared gradients instead of accumulating all past gradients. This moving average prevents the learning rate from becoming infinitesimally small, allowing the algorithm to continue making progress even after many iterations. The optimizer maintains g2, the exponentially weighted average of squared gradients, which is used to normalize the learning rate for each parameter. The decay parameter rho (typically 0.9) controls how quickly old gradients are forgotten. RMSProp is particularly effective for non-stationary problems and recurrent neural networks. The implementation supports both dictionary-based weights and numpy vectors.

Usage

Import from river.optim and use as an optimizer in any River model. Good default choice for problems with noisy or non-stationary gradients.

Code Reference

Source Location

• Repository: Online_ml_River
• File: river/optim/rms_prop.py

Signature

class RMSProp(optim.base.Optimizer):
    def __init__(self, lr=0.1, rho=0.9, eps=1e-8):
        ...

Import

from river import optim

I/O Contract

Inputs

Name	Type	Required	Description
lr	float	No (default=0.1)	Learning rate
rho	float	No (default=0.9)	Decay rate for moving average of squared gradients
eps	float	No (default=1e-8)	Small constant for numerical stability

Outputs

Name	Type	Description
optimizer	RMSProp	Configured optimizer instance ready for model training

Usage Examples

from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing

# Create RMSProp optimizer
optimizer = optim.RMSProp()

# Use with a linear model
dataset = datasets.Phishing()
model = (
    preprocessing.StandardScaler() |
    linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()

# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score)  # F1: 87.24%

# Custom parameters
optimizer = optim.RMSProp(
    lr=0.01,
    rho=0.95,
    eps=1e-7
)

model = linear_model.LogisticRegression(optimizer)

# Faster decay of gradient history
optimizer = optim.RMSProp(lr=0.1, rho=0.8)
model = linear_model.LogisticRegression(optimizer)

# Good for non-stationary problems
optimizer = optim.RMSProp()
model = linear_model.LogisticRegression(optimizer)

# Works well for recurrent patterns
from river import time_series
optimizer = optim.RMSProp(lr=0.05)

Related Pages

• Environment:Online_ml_River_Python_Runtime_Environment