Implementation:Pyro ppl Pyro SoftLaplace

Attribute	Value
Sources	pyro/distributions/softlaplace.py
Domains	Probabilistic Programming, Smooth Distributions, Laplace Approximation, Heavy-Tailed Distributions
Last Updated	2026-02-09

Overview

Description

The SoftLaplace distribution is a smooth, infinitely differentiable distribution with Laplace-like heavy tail behavior. It corresponds to the log-convex density defined by:

z = (value - loc) / scale
log_prob = log(2 / pi) - log(scale) - logaddexp(z, -z)

This distribution shares the key property of the Laplace distribution in that it has the heaviest possible tails (asymptotically) while still being log-convex. However, unlike the standard Laplace distribution, SoftLaplace is infinitely differentiable everywhere, including at the mode. This makes it particularly well-suited for constructing Laplace approximations, where the smoothness of the density is important for numerical optimization and second-order methods.

The distribution supports reparameterized sampling (has_rsample = True), making it compatible with gradient-based variational inference methods such as SVI with the ELBO objective.

Key properties:

• Mean: Equal to the loc parameter
• Variance: (pi / 2 * scale)^2
• Support: All real numbers
• CDF: (2 / pi) * arctan(exp(z)) where z = (value - loc) / scale
• Inverse CDF: loc + scale * log(tan(value * pi / 2))

Usage

SoftLaplace is useful whenever a Laplace-like distribution is desired but the non-differentiability of the Laplace density at its mode causes numerical issues. Common use cases include robust regression models, constructing smooth Laplace approximation posteriors, and situations where heavy-tailed priors with well-defined gradients are required.

Code Reference

Source Location

Property	Value
File	pyro/distributions/softlaplace.py
Module	pyro.distributions.softlaplace
Repository	pyro-ppl/pyro

Signature

class SoftLaplace(TorchDistribution):
    arg_constraints = {"loc": constraints.real, "scale": constraints.positive}
    support = constraints.real
    has_rsample = True

    def __init__(self, loc, scale, *, validate_args=None):
        ...

    def expand(self, batch_shape, _instance=None):
        ...

    def log_prob(self, value):
        ...

    def rsample(self, sample_shape=torch.Size()):
        ...

    def cdf(self, value):
        ...

    def icdf(self, value):
        ...

    @property
    def mean(self):
        ...

    @property
    def variance(self):
        ...

Import

from pyro.distributions import SoftLaplace

# Or from the module directly:
from pyro.distributions.softlaplace import SoftLaplace

I/O Contract

Constructor Parameters

Parameter	Type	Constraint	Description
loc	torch.Tensor or numeric	constraints.real	Location parameter (the mode and mean of the distribution)
scale	torch.Tensor or numeric	constraints.positive	Scale parameter (must be strictly positive)
validate_args	bool or None	--	Whether to validate input arguments. Default: None. Keyword-only argument.

Distribution Properties

Property	Type	Description
mean	torch.Tensor	Returns self.loc
variance	torch.Tensor	Returns (pi / 2 * self.scale) ** 2
has_rsample	bool	True -- supports reparameterized sampling
support	constraints.real	The entire real line
batch_shape	torch.Size	Shape determined by broadcasting loc and scale
event_shape	torch.Size	Empty (torch.Size([])), as this is a univariate distribution

Methods

Method	Return Type	Description
log_prob(value)	torch.Tensor	Computes log(2/pi) - log(scale) - logaddexp(z, -z) where z = (value - loc) / scale
rsample(sample_shape)	torch.Tensor	Draws reparameterized samples via inverse CDF transform of uniform noise
cdf(value)	torch.Tensor	Computes (2/pi) * arctan(exp(z))
icdf(value)	torch.Tensor	Computes loc + scale * log(tan(value * pi/2))
expand(batch_shape)	SoftLaplace	Returns a new SoftLaplace with expanded batch dimensions

Usage Examples

Basic Sampling and Log Probability

import torch
import pyro.distributions as dist

# Create a SoftLaplace distribution
d = dist.SoftLaplace(loc=torch.tensor(0.0), scale=torch.tensor(1.0))

# Draw samples (reparameterized)
samples = d.rsample(torch.Size([1000]))
print(samples.shape)  # torch.Size([1000])

# Compute log probabilities
log_probs = d.log_prob(samples)
print(log_probs.shape)  # torch.Size([1000])

# Check distributional properties
print(d.mean)      # tensor(0.)
print(d.variance)  # tensor(2.4674) ≈ (pi/2)^2

CDF and Inverse CDF

import torch
import pyro.distributions as dist

d = dist.SoftLaplace(loc=torch.tensor(0.0), scale=torch.tensor(2.0))

# Compute CDF values
x = torch.linspace(-5, 5, 100)
cdf_values = d.cdf(x)
print(cdf_values.min(), cdf_values.max())  # Near 0 and 1

# Inverse CDF (quantile function)
quantiles = torch.tensor([0.1, 0.25, 0.5, 0.75, 0.9])
values = d.icdf(quantiles)
print(values)  # Symmetric around loc=0

Using as a Prior in a Pyro Model

import torch
import pyro
import pyro.distributions as dist

def robust_regression(x, y=None):
    # SoftLaplace prior -- smooth yet heavy-tailed
    weight = pyro.sample("weight", dist.SoftLaplace(torch.tensor(0.0), torch.tensor(1.0)))
    bias = pyro.sample("bias", dist.SoftLaplace(torch.tensor(0.0), torch.tensor(1.0)))
    sigma = pyro.sample("sigma", dist.LogNormal(torch.tensor(0.0), torch.tensor(1.0)))

    mean = weight * x + bias
    with pyro.plate("data", x.shape[0]):
        pyro.sample("obs", dist.Normal(mean, sigma), obs=y)

Batched Distribution

import torch
import pyro.distributions as dist

# Batched parameters
locs = torch.tensor([0.0, 1.0, -1.0])
scales = torch.tensor([0.5, 1.0, 2.0])
d = dist.SoftLaplace(locs, scales)

print(d.batch_shape)  # torch.Size([3])
samples = d.rsample(torch.Size([100]))
print(samples.shape)  # torch.Size([100, 3])

Related Pages

• Laplace -- Standard Laplace distribution (non-smooth at mode); SoftLaplace provides a smooth alternative
• AsymmetricLaplace -- Asymmetric variant of the Laplace distribution
• SoftAsymmetricLaplace -- Smooth asymmetric Laplace variant
• Stable -- Another heavy-tailed distribution family available in Pyro
• Distributions_Init -- Central registry of all Pyro distributions