Heuristic:Online ml River Hoeffding Tree Grace Period Tuning

Knowledge Sources	River VFDT
Domains	Online_Learning, Decision_Trees, Parameter_Tuning
Last Updated	2026-02-08 16:00 GMT

Overview

Parameter tuning guide for Hoeffding Tree `grace_period` and `delta` controlling split timing and confidence.

Description

The `grace_period` and `delta` parameters are the two most impactful hyperparameters for Hoeffding Tree variants in River. `grace_period` controls how many observations a leaf must accumulate before attempting a split, while `delta` controls the confidence level of the Hoeffding bound used to determine whether a split is statistically justified. Together, they define the trade-off between fast adaptation and reliable split decisions. The standard HoeffdingTreeClassifier uses conservative defaults (grace_period=200, delta=1e-7), while the ARFClassifier uses aggressive defaults (grace_period=50, delta=0.01) to enable faster adaptation in ensemble settings.

Usage

Apply this heuristic when configuring any Hoeffding Tree variant in River, including `HoeffdingTreeClassifier`, `HoeffdingAdaptiveTreeClassifier`, `ARFClassifier`, and `HoeffdingTreeRegressor`. These parameters should be tuned based on the data stream characteristics: velocity, stationarity, and available memory.

The Insight (Rule of Thumb)

Action: Adjust `grace_period` based on stream velocity and stability.
Values:
- Fast/non-stationary streams: `grace_period=50`, `delta=0.01` (ARF defaults)
- Standard streams: `grace_period=200`, `delta=1e-7` (HoeffdingTree defaults)
- Stable/high-volume streams: `grace_period=500+`, `delta=1e-7`
Trade-off: Smaller grace_period = faster adaptation but more computational cost per observation and potentially less stable splits. Smaller delta = more conservative splits (longer wait) but higher confidence.
Tau (tie-breaking): Default `tau=0.05` across all variants. Below this threshold, a split is forced to break ties between equally good candidates.
Leaf prediction: Default `nba` (Naive Bayes Adaptive) is most accurate but slower; use `mc` (Majority Class) for maximum speed in high-velocity streams.

Reasoning

The Hoeffding Bound guarantees that the true mean of a random variable lies within a certain range of the observed mean with probability `1 - delta`. After observing `n` samples (the grace_period), the bound narrows, making split decisions more reliable. The key relationships are:

grace_period × delta interaction: A larger grace_period accumulates more evidence, compensating for a larger (less conservative) delta. This is why ARF uses grace_period=50 with delta=0.01 — the ensemble of 10 trees provides redundancy that individual tree confidence doesn't need to.
Memory management: `memory_estimate_period` controls how often memory is checked. HoeffdingTreeClassifier checks every 1,000,000 instances; ARFClassifier checks every 2,000,000 to reduce overhead in ensemble settings.
max_size: Default 100 MiB for classification, which limits tree growth. If trees are being pruned due to memory, increase this or reduce grace_period to create fewer, more significant splits.

The theoretical guarantee of Hoeffding Trees is that the output is asymptotically identical to a batch decision tree learner with infinite data, provided the grace_period is large enough and delta is small enough.

Code Evidence

HoeffdingTreeClassifier defaults from `river/tree/hoeffding_tree_classifier.py:130-148`:

def __init__(
    self,
    grace_period: int = 200,
    max_depth: int | None = None,
    split_criterion: str = "info_gain",
    delta: float = 1e-7,
    tau: float = 0.05,
    leaf_prediction: str = "nba",
    nb_threshold: int = 0,
    nominal_attributes: list | None = None,
    splitter: Splitter | None = None,
    binary_split: bool = False,
    min_branch_fraction: float = 0.01,
    max_share_to_split: float = 0.99,
    max_size: float = 100.0,
    memory_estimate_period: int = 1000000,
    ...
):

ARFClassifier aggressive defaults from `river/forest/adaptive_random_forest.py:593-617`:

def __init__(
    self,
    n_models: int = 10,
    max_features: bool | str | int = "sqrt",
    lambda_value: int = 6,
    ...
    grace_period: int = 50,
    ...
    delta: float = 0.01,
    tau: float = 0.05,
    leaf_prediction: str = "nba",
    ...
    max_size: float = 100.0,
    memory_estimate_period: int = 2_000_000,
    ...
):

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment