Implementation:Online ml River Optim AdaGrad
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Optimization |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
AdaGrad is an adaptive learning rate optimizer that adapts the learning rate for each parameter based on historical gradient information.
Description
AdaGrad (Adaptive Gradient Algorithm) adapts the learning rate individually for each parameter, performing smaller updates for frequently occurring features and larger updates for infrequent features. It accumulates the squared gradients over time and uses this accumulation to scale the learning rate for each parameter. The learning rate for each parameter decreases at a rate inversely proportional to the square root of the sum of all historical squared gradients. This makes AdaGrad particularly well-suited for sparse data and features that occur infrequently, though the continual accumulation can cause the learning rate to shrink too aggressively in long training runs.
Usage
Import from river.optim and use as an optimizer in any River model. Best suited for problems with sparse features or when different features have vastly different frequencies.
Code Reference
Source Location
- Repository: Online_ml_River
- File: river/optim/ada_grad.py
Signature
class AdaGrad(optim.base.Optimizer):
def __init__(self, lr=0.1, eps=1e-8):
...
Import
from river import optim
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| lr | float | No (default=0.1) | Learning rate |
| eps | float | No (default=1e-8) | Small constant for numerical stability |
Outputs
| Name | Type | Description |
|---|---|---|
| optimizer | AdaGrad | Configured optimizer instance ready for model training |
Usage Examples
from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
# Create AdaGrad optimizer
optimizer = optim.AdaGrad()
# Use with a linear model
dataset = datasets.Phishing()
model = (
preprocessing.StandardScaler() |
linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()
# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score) # F1: 88.01%
# Custom parameters
optimizer = optim.AdaGrad(lr=0.05, eps=1e-6)
model = linear_model.LogisticRegression(optimizer)
# Good for sparse features
from river import feature_extraction
model = (
feature_extraction.BagOfWords() |
linear_model.LogisticRegression(optim.AdaGrad())
)