Principle:Pyro ppl Pyro Learnable Parameters
| Knowledge Sources | |
|---|---|
| Domains | Probabilistic_Programming, Optimization |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A principle for declaring optimizable parameters within probabilistic programs that are learned during variational inference.
Description
In variational inference, the guide (variational distribution) contains learnable parameters — values that are optimized to minimize the KL divergence between the guide and the true posterior. Unlike sample sites (which are random variables), parameters are deterministic values stored in a global parameter store and updated by gradient-based optimization during SVI.
Parameters support constraints (e.g., positivity for scale parameters) via PyTorch's constraint system. The parameter store manages an unconstrained representation internally and applies the constraint transform when the parameter is accessed. Parameters can also be subsampled when used inside plate contexts, supporting mini-batch training with large parameter tensors.
The parameter primitive is the bridge between Pyro's probabilistic programming model and PyTorch's optimization framework: parameters are the leaf nodes that accumulate gradients during the backward pass.
Usage
Use learnable parameters in variational guides to define the free parameters of the variational distribution (e.g., the mean and standard deviation of a Normal guide). Also used in models when point-estimated parameters are desired (though MAP estimation via AutoDelta guides is often preferred). Parameters are typically initialized once and updated across many SVI steps.
Theoretical Basis
In variational inference, we optimize parameters of the guide to minimize:
Parameters are updated by gradient descent:
Constrained parameters use reparameterization: if must satisfy a constraint (e.g., ), we optimize an unconstrained and apply a transform (e.g., softplus).