Implementation:Online ml River ModelSelection SuccessiveHalving

Knowledge Sources	Online_ml_River
Domains	Online_Learning, Model_Selection, Hyperparameter_Optimization
Last Updated	2026-02-08 16:00 GMT

Overview

Successive Halving is an efficient online model selection algorithm that progressively eliminates poorly performing models at predefined rungs to find the best hyperparameter configuration within a fixed budget.

Description

Successive Halving trains multiple model candidates in parallel, periodically eliminating the worst performers while continuing to train the best ones. At each rung (checkpoint), the algorithm sorts models by their metric performance and discards approximately half (controlled by eta parameter). The remaining models receive more training iterations before the next elimination round. The budget parameter controls total model updates across all candidates. Rung positions are calculated to ensure the budget is fully utilized. The algorithm maintains running performance metrics for each model and selects the current best for predictions at any time.

Usage

Use Successive Halving for hyperparameter tuning when you have a fixed computational budget and multiple model configurations to compare. It is more efficient than grid search or random search for online learning. Set budget to approximately 2*k*n/eta where k is the number of configurations and n is dataset size. Use eta=2 for aggressive elimination or higher values to focus on fewer top models. The verbose flag shows elimination progress. Most effective when models have clearly different performance levels that emerge early in training.

Code Reference

Source Location

Repository: Online_ml_River
File: river/model_selection/sh.py

Signature

class SuccessiveHalvingRegressor(
    models,
    metric: metrics.base.Metric,
    budget: int,
    eta=2,
    verbose=False,
    **print_kwargs,
)

class SuccessiveHalvingClassifier(
    models,
    metric: metrics.base.Metric,
    budget: int,
    eta=2,
    verbose=False,
    **print_kwargs,
)

Import

from river import model_selection

I/O Contract

Input
Parameter	Type	Description
models	list	List of model instances to compare
x	dict	Feature dictionary for training/prediction
y	Any	Target value (regression or classification)

Output
Method	Return Type	Description
predict_one(x)	Any	Prediction from current best model
predict_proba_one(x)	dict	Class probabilities (classifier only)
learn_one(x, y)	None	Updates active models and eliminates poor ones
best_model	Estimator	Returns the currently best performing model

Usage Examples

from river import datasets
from river import evaluate
from river import linear_model
from river import metrics
from river import model_selection
from river import optim
from river import preprocessing
from river import utils

# Define the base model
model = (
    preprocessing.StandardScaler() |
    linear_model.LinearRegression(intercept_lr=.1)
)

# Create parameter grid
models = utils.expand_param_grid(model, {
    'LinearRegression': {
        'optimizer': [
            (optim.SGD, {'lr': [.1, .01, .005]}),
            (optim.Adam, {'beta_1': [.01, .001], 'lr': [.1, .01, .001]}),
        ]
    }
})

# Apply successive halving
sh = model_selection.SuccessiveHalvingRegressor(
    models,
    metric=metrics.MAE(),
    budget=2000,
    eta=2,
    verbose=True
)

# Evaluate
result = evaluate.progressive_val_score(
    dataset=datasets.TrumpApproval(),
    model=sh,
    metric=metrics.MAE()
)
print(f"Best model: {sh.best_model}")

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment