Principle:Pyro ppl Pyro Conditional Independence

Knowledge Sources	Pyro: Deep Universal Probabilistic Programming Stochastic Variational Inference Pyro Tensor Shapes
Domains	Probabilistic_Programming, Bayesian_Inference
Last Updated	2026-02-09 00:00 GMT

Overview

A principle for declaring conditional independence structure in probabilistic models, enabling vectorized computation and correct subsampling of large datasets.

Description

Conditional independence is a core concept in probabilistic modeling: observations are often conditionally independent given the model parameters. For example, in a regression model, each data point's likelihood is independent of other data points given the regression coefficients.

In Pyro, conditional independence is declared using plate contexts. A plate groups together sample sites that are conditionally independent, serving three purposes:

1. Correctness: Ensures the ELBO and gradient estimators correctly account for independence structure
2. Vectorization: Enables efficient batched computation over independent samples
3. Subsampling: Allows training on mini-batches with correct scaling of the ELBO — the plate automatically scales log-probabilities by the ratio of the full dataset size to the subsample size

Plates can be nested to represent multi-level independence structures (e.g., words within documents within a corpus in topic models). Proper plate annotation is essential for correct inference, especially with parallel enumeration of discrete variables.

Usage

Use this principle whenever your model has repeated, conditionally independent structure — which includes virtually all models applied to datasets with multiple observations. Plates are required for mini-batch training with SVI, for parallel enumeration with TraceEnum_ELBO, and for correct posterior predictive sampling with the Predictive class.

Theoretical Basis

Given N observations $x_1,\dots ,x_N$ that are conditionally independent given latent variables $\mathbf{𝐳}$:

$p(\mathbf{𝐱}|\mathbf{𝐳})=\prod _{i=1}^{N}p(x_i|\mathbf{𝐳})$

The log-likelihood decomposes as a sum:

$\log p(\mathbf{𝐱}|\mathbf{𝐳})=\sum _{i=1}^N\log p(x_i|\mathbf{𝐳})$

For subsampling with a mini-batch of size M:

$\log p(\mathbf{𝐱}|\mathbf{𝐳})\approx \frac{N}{M}\sum _{i\in \text{batch}}\log p(x_i|\mathbf{𝐳})$

Pseudo-code:

# Abstract conditional independence pattern
def model(data):
    z = sample("z", Prior(...))
    with plate("data", size=N, subsample_size=M):
        sample("obs", Likelihood(z), obs=data_batch)

Related Pages

Implemented By

• Implementation:Pyro_ppl_Pyro_Pyro_Plate