Principle:Pyro ppl Pyro Time Series Forecasting
| Knowledge Sources | |
|---|---|
| Domains | Time Series, Gaussian Processes, Forecasting |
| Last Updated | 2026-02-09 09:00 GMT |
Overview
GP and HMM-based time series forecasting combines Gaussian processes for modeling smooth temporal trends with Hidden Markov Models for capturing regime changes, providing probabilistic forecasts with calibrated uncertainty.
Description
Time series forecasting requires models that capture temporal patterns while quantifying uncertainty about future observations. Probabilistic approaches are essential because:
- Point forecasts are often insufficient for decision-making.
- Uncertainty grows with the forecast horizon.
- External factors may cause regime changes or structural breaks.
Gaussian Process time series models: GPs provide a non-parametric, Bayesian approach to modeling temporal functions. The key idea is to place a GP prior over the time-varying function and condition on observed data to obtain a posterior distribution over future values. The kernel function encodes assumptions about the temporal structure:
- Matern kernel: Captures smooth but non-infinitely-differentiable trends.
- Periodic kernel: Captures regular seasonal patterns.
- Linear kernel: Captures long-term trends.
- Kernel composition: Products and sums of kernels capture complex patterns (e.g., growing periodic signal = linear * periodic).
HMM-based forecasting: For time series with discrete regime changes (e.g., bull/bear markets, outbreak/endemic phases), Hidden Markov Models capture switching dynamics. Each regime has its own observation distribution, and transitions between regimes are governed by a Markov chain.
Combined approaches use GPs for smooth within-regime dynamics and HMMs for regime switching, or use hierarchical models where GP hyperparameters are shared across related time series.
Practical forecasting with these models requires handling:
- Missing data: Not all time steps may be observed.
- Irregular spacing: Observations may not be evenly spaced.
- Multiple related series: Hierarchical models share information across series.
- External covariates: Regressors like temperature or holidays affect the forecast.
Usage
Use GP/HMM time series forecasting when:
- Probabilistic forecasts with uncertainty quantification are needed.
- The time series exhibits smooth trends, periodicity, or regime changes.
- Forecasting with limited historical data where Bayesian priors are valuable.
- Working with irregularly sampled or partially observed time series.
- Combining multiple related time series in a hierarchical framework.
Theoretical Basis
GP regression for time series:
# Prior: f ~ GP(m(t), k(t, t'))
# Observation: y_t = f(t) + epsilon, epsilon ~ Normal(0, sigma^2)
# Given observations y at times T_train:
# Posterior at test times T_test:
# f* | y ~ Normal(mu*, Sigma*)
# mu* = K(T_test, T_train) [K(T_train, T_train) + sigma^2 I]^{-1} y
# Sigma* = K(T_test, T_test) - K(T_test, T_train) [K(T_train, T_train) + sigma^2 I]^{-1} K(T_train, T_test)
# Marginal likelihood for hyperparameter optimization:
# log p(y | theta) = -0.5 * y^T K_y^{-1} y - 0.5 * log|K_y| - n/2 * log(2*pi)
# where K_y = K(T_train, T_train) + sigma^2 I, theta = kernel hyperparameters
Kernel composition for temporal patterns:
# Trend + seasonality + noise:
# k(t, t') = k_trend(t, t') + k_seasonal(t, t') + k_noise(t, t')
# k_trend: Matern-5/2 with long lengthscale
# k_seasonal: Periodic kernel * Matern (decaying periodicity)
# k_noise: White noise kernel (sigma^2 * delta(t, t'))
# Periodic kernel:
# k_periodic(t, t') = sigma^2 * exp(-2 * sin^2(pi * |t-t'| / period) / l^2)
HMM regime-switching model:
# K regimes with different dynamics:
# z_t ~ Categorical(A[z_{t-1}]) (regime transition)
# y_t | z_t=k ~ Normal(mu_k + phi_k * y_{t-1}, sigma_k^2) (AR(1) per regime)
# Example: economic regimes
# Regime 1 (expansion): high mu, low sigma
# Regime 2 (recession): low mu, high sigma
# Forecast: marginalize over future regime paths
# p(y_{T+h} | y_{1:T}) = sum over regime paths z_{T+1:T+h}:
# p(z_{T+1:T+h} | z_T) * p(y_{T+h} | z_{T+h}, y_{T+h-1})
BART (Bayesian Additive Regression Trees) for time series:
# Decompose time series into additive components:
# y_t = f_trend(t) + f_seasonal(t) + f_covariates(x_t) + epsilon_t
# Each component modeled separately:
# f_trend: GP with long-range kernel
# f_seasonal: periodic component
# f_covariates: regression on external variables
# Bayesian inference over all components simultaneously
# Provides decomposition into interpretable components