Implementation:Online ml River Optim Schedulers

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Optimization
Last Updated	2026-02-08 16:00 GMT

Overview

Learning rate schedulers that dynamically adjust the learning rate during training, including constant, inverse scaling, and optimal schedules.

Description

Learning rate schedulers control how the learning rate changes over time during training. The module provides three scheduler types: Constant (maintains a fixed learning rate), InverseScaling (reduces learning rate using a power schedule proportional to 1/(t+1)^power), and Optimal (implements Léon Bottou's optimal learning schedule that adapts based on the loss function and a regularization parameter). Schedulers implement the get(t) method that returns the learning rate at iteration t. The Optimal scheduler is particularly sophisticated, computing an initial learning rate based on typical weight values and the loss gradient, then decaying it according to a schedule that balances convergence speed with stability. These schedulers can be used with any optimizer that accepts a Scheduler object.

Usage

Import from river.optim.schedulers to create learning rate schedules. Pass scheduler instances to optimizers that support them, such as AMSGrad.

Code Reference

Source Location

• Repository: Online_ml_River
• File: river/optim/schedulers.py

Signature

class Constant(optim.base.Scheduler):
    def __init__(self, learning_rate: int | float):
        ...
    def get(self, t):
        ...

class InverseScaling(optim.base.Scheduler):
    def __init__(self, learning_rate: float, power=0.5):
        ...
    def get(self, t):
        ...

class Optimal(optim.base.Scheduler):
    def __init__(self, loss: optim.losses.Loss, alpha=1e-4):
        ...
    def get(self, t):
        ...

Import

from river import optim

I/O Contract

Inputs

Name	Type	Required	Description
learning_rate	int or float	Yes (Constant, InverseScaling)	Initial learning rate
power	float	No (default=0.5)	Power for inverse scaling decay
loss	optim.losses.Loss	Yes (Optimal)	Loss function for optimal schedule computation
alpha	float	No (default=1e-4)	Regularization parameter for optimal schedule
t	int	Yes (get method)	Current iteration number

Outputs

Name	Type	Description
learning_rate	float	Learning rate at iteration t (from get method)

Usage Examples

from river import optim
from river import linear_model
from river import datasets
from river import evaluate
from river import metrics
from river import preprocessing

# Constant learning rate
scheduler = optim.schedulers.Constant(learning_rate=0.1)
print(scheduler.get(0))    # 0.1
print(scheduler.get(100))  # 0.1

# Inverse scaling: lr = initial_lr / (t + 1)^power
scheduler = optim.schedulers.InverseScaling(learning_rate=1.0, power=0.5)
print(scheduler.get(0))    # 1.0
print(scheduler.get(3))    # 0.5
print(scheduler.get(15))   # 0.25

# Optimal schedule based on loss function
loss = optim.losses.Hinge()
scheduler = optim.schedulers.Optimal(loss=loss, alpha=1e-4)
print(scheduler.get(0))
print(scheduler.get(100))

# Use with optimizer
dataset = datasets.Phishing()

# Constant schedule (explicit)
optimizer = optim.SGD(lr=optim.schedulers.Constant(0.01))
model = linear_model.LogisticRegression(optimizer)

# Inverse scaling schedule
optimizer = optim.SGD(
    lr=optim.schedulers.InverseScaling(learning_rate=0.1, power=0.5)
)
model = (
    preprocessing.StandardScaler() |
    linear_model.LogisticRegression(optimizer)
)

# Optimal schedule
loss = optim.losses.Log()
optimizer = optim.SGD(
    lr=optim.schedulers.Optimal(loss=loss, alpha=1e-3)
)
model = linear_model.LogisticRegression(optimizer)

# With AMSGrad (which supports schedulers)
scheduler = optim.schedulers.InverseScaling(0.1, power=0.3)
optimizer = optim.AMSGrad(lr=scheduler)
model = linear_model.LogisticRegression(optimizer)

Related Pages

• Environment:Online_ml_River_Python_Runtime_Environment