Heuristic:Fastai Fastbook Random Forest Defaults
| Knowledge Sources | |
|---|---|
| Domains | Tabular, Machine_Learning |
| Last Updated | 2026-02-09 17:00 GMT |
Overview
Practical defaults for Random Forests: `n_estimators=40`, `max_samples=200_000`, `max_features=0.5`, `min_samples_leaf=5`.
Description
Random Forests are ensembles of decision trees where each tree is trained on a random subset of rows and considers a random subset of columns at each split. The Fastbook provides a practical wrapper function `rf()` with carefully chosen defaults that balance accuracy, speed, and robustness. These defaults are generally insensitive to the specific dataset, making Random Forests an excellent baseline model.
Usage
Use these defaults as a starting point for any tabular regression or classification problem. The key insight from the Fastbook is that Random Forests are not very sensitive to hyperparameter choices, so the defaults work well in most cases. Adjust only when:
- Very large datasets (> 200K rows): `max_samples=200_000` caps training set per tree for speed
- Diminishing returns: More trees (`n_estimators`) always help but with diminishing returns
- Overfitting: Increase `min_samples_leaf` to reduce tree depth
The Insight (Rule of Thumb)
- Action: Use the `rf()` wrapper function or set these parameters directly in `RandomForestRegressor`.
- Values:
- `n_estimators=40`: Number of trees. More is always better but with diminishing returns. Set as high as your compute budget allows.
- `max_samples=200_000`: Rows per tree. Keep default unless dataset is much smaller. For datasets >200K rows, this speeds training with minimal accuracy loss.
- `max_features=0.5`: Columns sampled per split (50% of features). Using fewer features with more trees produces lower error.
- `min_samples_leaf=5`: Minimum samples to make a leaf. Prevents overly deep trees.
- `n_jobs=-1`: Use all CPU cores for parallel tree construction.
- Trade-off: More trees and higher max_samples increase accuracy but linearly increase training time.
Reasoning
The sklearn documentation shows that using fewer features per split (`max_features=0.5`) combined with more trees produces the lowest error — trees with fewer features are more diverse, and diversity is what makes ensembles effective. The `max_samples` parameter sub-samples the dataset for each tree; for datasets larger than 200K rows, capping at 200K significantly speeds training with very small accuracy impact because each tree already gets a representative sample.
The Fastbook explicitly states: "Random forests are not very sensitive to hyperparameter choices... `max_features=0.5` and `min_samples_leaf=4` generally work well, although sklearn's defaults are also fine."
Code Evidence
Random Forest wrapper function from `09_tabular.md:617-621`:
def rf(xs, y, n_estimators=40, max_samples=200_000,
max_features=0.5, min_samples_leaf=5, **kwargs):
return RandomForestRegressor(n_jobs=-1, n_estimators=n_estimators,
max_samples=max_samples, max_features=max_features,
min_samples_leaf=min_samples_leaf, oob_score=True).fit(xs, y)
Hyperparameter insensitivity guidance from `09_tabular.md:638`:
Random forests are not very sensitive to hyperparameter choices, like
max_features. You can set n_estimators to as high a number as you have time
for. max_samples can usually be left at its default, unless you have over
200,000 data points, in which case setting it to 200,000 will make it
faster to train with little impact on accuracy. max_features=0.5 and
min_samples_leaf=4 generally work well.