Implementation:Online ml River Optim AdaDelta
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Optimization |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
AdaDelta is an adaptive learning rate optimizer that does not require manual tuning of a learning rate hyperparameter.
Description
AdaDelta is an extension of AdaGrad that addresses its continually decreasing learning rate problem. Instead of accumulating all past squared gradients, AdaDelta restricts the window of accumulated past gradients to a fixed size. Rather than storing this window of past gradients, it recursively maintains a decaying average of past squared gradients. The learning rate is computed using the ratio of RMS of parameter updates to RMS of gradients, making it a hyperparameter-free method that adapts the learning rate based on the history of gradients and updates.
Usage
Import from river.optim and use as an optimizer in any River model. Particularly useful when you want adaptive learning rates without manually tuning the learning rate parameter.
Code Reference
Source Location
- Repository: Online_ml_River
- File: river/optim/ada_delta.py
Signature
class AdaDelta(optim.base.Optimizer):
def __init__(self, rho=0.95, eps=1e-8):
...
Import
from river import optim
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| rho | float | No (default=0.95) | Decay rate for accumulating squared gradients and updates |
| eps | float | No (default=1e-8) | Small constant for numerical stability |
Outputs
| Name | Type | Description |
|---|---|---|
| optimizer | AdaDelta | Configured optimizer instance ready for model training |
Usage Examples
from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
# Create AdaDelta optimizer (no learning rate needed!)
optimizer = optim.AdaDelta()
# Use with a linear model
dataset = datasets.Phishing()
model = (
preprocessing.StandardScaler() |
linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()
# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score) # F1: 80.56%
# Custom decay rate
optimizer = optim.AdaDelta(rho=0.9, eps=1e-6)
model = linear_model.LogisticRegression(optimizer)