Implementation:Online ml River Optim SGD
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Optimization |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
SGD (Stochastic Gradient Descent) is the fundamental optimizer that updates weights proportional to the negative gradient with a fixed learning rate.
Description
Stochastic Gradient Descent is the most basic and foundational optimization algorithm for online learning. It performs a simple weight update by moving in the direction opposite to the gradient, scaled by a learning rate. The update rule is straightforward: w = w - lr * gradient. Despite its simplicity, SGD is surprisingly effective and serves as the baseline against which other optimizers are compared. It works well when the learning rate is properly tuned and is often preferred for its simplicity, low memory footprint, and good generalization properties. The implementation supports both dictionary-based weights and numpy vectors, making it versatile for different model types. SGD with a well-tuned learning rate schedule often matches or exceeds the performance of more sophisticated optimizers.
Usage
Import from river.optim and use as an optimizer in any River model. Good baseline choice and often effective with proper learning rate tuning.
Code Reference
Source Location
- Repository: Online_ml_River
- File: river/optim/sgd.py
Signature
class SGD(optim.base.Optimizer):
def __init__(self, lr=0.01) -> None:
...
Import
from river import optim
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| lr | float | No (default=0.01) | Learning rate |
Outputs
| Name | Type | Description |
|---|---|---|
| optimizer | SGD | Configured optimizer instance ready for model training |
Usage Examples
from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
# Create SGD optimizer
optimizer = optim.SGD(0.1)
# Use with a linear model
dataset = datasets.Phishing()
model = (
preprocessing.StandardScaler() |
linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()
# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score) # F1: 87.85%
# Different learning rates
optimizer = optim.SGD(lr=0.01) # Smaller steps
model = linear_model.LogisticRegression(optimizer)
optimizer = optim.SGD(lr=0.5) # Larger steps
model = linear_model.LogisticRegression(optimizer)
# With learning rate scheduler
from river.optim import schedulers
scheduler = schedulers.InverseScaling(learning_rate=0.1, power=0.5)
optimizer = optim.SGD(lr=scheduler)
model = linear_model.LogisticRegression(optimizer)
# Simple and effective baseline
optimizer = optim.SGD(0.01)
classifier = linear_model.LogisticRegression(optimizer)
regressor = linear_model.LinearRegression(optimizer)
# Can be wrapped with Averager for stability
optimizer = optim.Averager(optim.SGD(0.1), start=100)
model = linear_model.LogisticRegression(optimizer)