Principle:LaurentMazare Tch rs VarStore Initialization
| Knowledge Sources | |
|---|---|
| Domains | Deep_Learning, Parameter_Management |
| Last Updated | 2026-02-08 14:00 GMT |
Overview
Mechanism for creating a centralized, device-aware parameter store that manages all trainable variables for a neural network model.
Description
A Variable Store (VarStore) is a container that holds all learnable parameters of a neural network in a single, cohesive structure. It provides a hierarchical naming system (via paths) for parameter registration, enables transparent device placement (CPU/CUDA), and serves as the interface between model parameters and optimizers. The VarStore pattern decouples parameter storage from model architecture, allowing serialization, device migration, and optimizer binding to be handled orthogonally.
Usage
Use this principle at the start of any model construction pipeline. A VarStore must be created before defining any neural network layers, as layers register their parameters (weights, biases) into the VarStore via its path system. The VarStore's device determines where all parameters will be allocated.
Theoretical Basis
The VarStore pattern implements a parameter registry with:
- Device binding: All parameters allocated on a single device (CPU or CUDA)
- Hierarchical naming: Parameters organized via path separators (e.g., "layer1.weight")
- Trainability tracking: Separate tracking of trainable vs. frozen variables
- Thread safety: Arc<Mutex<Variables>> for concurrent access
VarStore Architecture:
VarStore(device)
└── Variables
├── named_variables: HashMap<String, Tensor>
└── trainable_variables: Vec<Tensor>
Path system: vs.root() / "layer1" / "weight" → "layer1.weight"