Heuristic:Fastai Fastbook Batch Size Selection
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Deep_Learning |
| Last Updated | 2026-02-09 17:00 GMT |
Overview
Batch size selection: 64 for images (fastai default), larger batches (1024) for tabular data which uses less GPU memory.
Description
Batch size determines how many data items are processed together in each training step. The optimal batch size depends on the data modality and available GPU memory. The Fastbook uses different batch sizes across its workflows: the default of 64 for image data, and much larger batch sizes (1024) for tabular data where individual samples consume far less GPU memory than images.
Usage
Use this heuristic when creating `DataLoaders` for any training workflow. Consider adjusting batch size when:
- CUDA out of memory: Reduce batch size to fit in GPU VRAM
- Tabular data: Increase batch size since tabular samples are small
- Training speed: Larger batches improve GPU utilization but may require LR adjustment
The Insight (Rule of Thumb)
- Action: Set `bs=` parameter when creating DataLoaders.
- Value:
- Images: `bs=64` (fastai default) — adjust down for large images or limited VRAM
- Tabular: `bs=1024` — tabular data uses far less GPU memory per sample
- Text: `bs=64` or `bs=128` — depends on sequence length
- Trade-off: Larger batch sizes are more efficient (better GPU utilization) but may generalize worse. Smaller batches provide more noise which can help regularization.
Reasoning
The DataLoader provides batches of items to the GPU. fastai defaults to 64 items per batch. The Fastbook explicitly notes that "tabular models and data usually don't need as much GPU RAM, so we can use larger batch sizes" and demonstrates `dls = to_nn.dataloaders(1024)` for the tabular deep learning model. For image data, 64 items of 224x224 RGB images consume significantly more memory than 1024 tabular rows, which explains the different defaults.
Code Evidence
Default batch size mentioned from `02_production` notebook:
DataLoader is a class that provides batches of a few items at a time to the GPU...
fastai will give you 64 (by default) items at a time
Image data with default batch size from `07_sizing_and_tta.md:50`:
dls = dblock.dataloaders(path, bs=64)
Tabular data with large batch size from `09_tabular.md:1324-1327`:
# Tabular models and data usually don't need as much GPU memory,
# so we can use larger batch sizes:
dls = to_nn.dataloaders(1024)