Implementation:Online ml River Optim AdaMax

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Optimization
Last Updated	2026-02-08 16:00 GMT

Overview

AdaMax is a variant of Adam based on the infinity norm that can be more stable than Adam in some cases.

Description

AdaMax is a variation of the Adam optimizer that replaces the L2 norm-based second moment with the L-infinity norm (maximum absolute value). Instead of computing an exponentially weighted average of squared gradients, AdaMax tracks the exponentially weighted infinity norm of past gradients. This makes the algorithm less sensitive to large gradients and can provide more stable behavior in certain scenarios. The update rule uses the first moment estimate divided by the infinity norm of past gradients, with bias correction applied only to the first moment. AdaMax can be particularly effective when gradients are sparse or when dealing with embeddings.

Usage

Import from river.optim and use as an optimizer in any River model. Consider using AdaMax when Adam shows instability or when working with sparse gradients.

Code Reference

Source Location

• Repository: Online_ml_River
• File: river/optim/ada_max.py

Signature

class AdaMax(optim.base.Optimizer):
    def __init__(self, lr=0.1, beta_1=0.9, beta_2=0.999, eps=1e-8):
        ...

Import

from river import optim

I/O Contract

Inputs

Name	Type	Required	Description
lr	float	No (default=0.1)	Learning rate
beta_1	float	No (default=0.9)	Exponential decay rate for first moment estimates
beta_2	float	No (default=0.999)	Exponential decay rate for exponentially weighted infinity norm
eps	float	No (default=1e-8)	Small constant for numerical stability

Outputs

Name	Type	Description
optimizer	AdaMax	Configured optimizer instance ready for model training

Usage Examples

from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing

# Create AdaMax optimizer
optimizer = optim.AdaMax()

# Use with a linear model
dataset = datasets.Phishing()
model = (
    preprocessing.StandardScaler() |
    linear_model.LogisticRegression(optimizer)
)
metric = metrics.F1()

# Evaluate
score = evaluate.progressive_val_score(dataset, model, metric)
print(score)  # F1: 87.61%

# Custom parameters
optimizer = optim.AdaMax(
    lr=0.01,
    beta_1=0.95,
    beta_2=0.99,
    eps=1e-7
)

model = linear_model.LogisticRegression(optimizer)

Related Pages

• Environment:Online_ml_River_Python_Runtime_Environment