Implementation:Online ml River Optim Newton

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Optimization
Last Updated	2026-02-08 16:00 GMT

Overview

Newton optimizer (Online Newton Step) uses second-order information by maintaining an inverse Hessian approximation for potentially faster convergence.

Description

The Newton optimizer, also known as Online Newton Step (ONS), uses second-order gradient information by maintaining an approximation of the inverse Hessian matrix. Unlike first-order methods that only use gradient information, Newton's method incorporates curvature information from the Hessian (second derivatives) to make more informed updates. The implementation efficiently maintains the inverse Hessian using the Sherman-Morrison formula, which allows for incremental updates without expensive matrix inversions. This second-order information enables the optimizer to make better decisions about step sizes in different directions, potentially converging much faster than first-order methods. However, it requires more computation and memory per step.

Usage

Import from river.optim and use as an optimizer in any River model. Best suited for problems where fast convergence is critical and computational cost per iteration is acceptable.

Code Reference

Source Location

• Repository: Online_ml_River
• File: river/optim/newton.py

Signature

def sherman_morrison(A_inv: dict, u: dict, v: dict) -> dict:
    """Sherman–Morrison formula for efficient inverse Hessian updates."""
    ...

class Newton(optim.base.Optimizer):
    def __init__(self, lr=0.1, eps=1e-5):
        ...

Import

from river import optim

I/O Contract

Inputs

Name	Type	Required	Description
lr	float	No (default=0.1)	Learning rate
eps	float	No (default=1e-5)	Initial diagonal value for inverse Hessian (regularization)

Outputs

Name	Type	Description
optimizer	Newton	Configured optimizer instance ready for model training

Usage Examples

from river import datasets
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
from river import evaluate

# Create Newton optimizer
optimizer = optim.Newton()

# Use with a linear model
dataset = datasets.Phishing()
model = (
    preprocessing.StandardScaler() |
    linear_model.LogisticRegression(optimizer)
)

# For faster convergence on smaller problems
optimizer = optim.Newton(lr=0.5, eps=1e-4)
model = linear_model.LogisticRegression(optimizer)

# Newton method can converge very quickly
from river import stream

X_y = [
    ({'x1': 1, 'x2': 2}, 0),
    ({'x1': 3, 'x2': 4}, 1),
    ({'x1': 5, 'x2': 6}, 1),
]

model = linear_model.LogisticRegression(optim.Newton())

for x, y in X_y:
    model.learn_one(x, y)
    y_pred = model.predict_one(x)
    print(f"Prediction: {y_pred}, True: {y}")

# Note: More computationally expensive than first-order methods
# Use when fast convergence is more important than per-iteration cost

Related Pages

• Environment:Online_ml_River_Python_Runtime_Environment