Implementation:Online ml River Optim Momentum
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Optimization |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
Momentum optimizer accelerates gradient descent by accumulating a velocity vector in directions of persistent reduction in the objective.
Description
Momentum is an extension of stochastic gradient descent that helps accelerate convergence and dampen oscillations. It does this by adding a fraction (rho) of the previous update vector to the current gradient. This accumulation creates a "velocity" that builds up in directions where the gradient consistently points, allowing the optimizer to move faster through flat regions and smooth out noisy gradients. The momentum parameter rho (typically 0.9) controls how much of the previous update is retained. Higher values give more weight to past gradients, creating stronger momentum effects. This technique is particularly effective in navigating ravines and areas where the surface curves more steeply in one dimension than another.
Usage
Import from river.optim and use as an optimizer in any River model. Particularly effective when the loss surface has ravines or noisy gradients.
Code Reference
Source Location
- Repository: Online_ml_River
- File: river/optim/momentum.py
Signature
class Momentum(optim.base.Optimizer):
def __init__(self, lr=0.1, rho=0.9):
...
Import
from river import optim
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| lr | float | No (default=0.1) | Learning rate |
| rho | float | No (default=0.9) | Momentum parameter (fraction of previous update to retain) |
Outputs
| Name | Type | Description |
|---|---|---|
| optimizer | Momentum | Configured optimizer instance ready for model training |
Usage Examples
from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
# Create Momentum optimizer
optimizer = optim.Momentum()
# Use with a linear model
dataset = datasets.Phishing()
model = (
preprocessing.StandardScaler() |
linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()
# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score) # F1: 84.09%
# Custom parameters
optimizer = optim.Momentum(lr=0.01, rho=0.95)
model = linear_model.LogisticRegression(optimizer)
# Lower momentum for more responsive updates
optimizer = optim.Momentum(lr=0.05, rho=0.8)
model = linear_model.LogisticRegression(optimizer)
# Higher momentum for smoother trajectories
optimizer = optim.Momentum(lr=0.1, rho=0.99)
model = linear_model.LogisticRegression(optimizer)