Principle:Rapidsai Cuml Time Series Forecasting

Knowledge Sources	Holt 1957 - Forecasting Seasonals and Trends by Exponentially Weighted Moving Averages Winters 1960 - Forecasting Sales by Exponentially Weighted Moving Averages Box & Jenkins 1970 - Time Series Analysis: Forecasting and Control Kwiatkowski et al. 1992 - Testing the Null Hypothesis of Stationarity (KPSS)
Domains	Machine_Learning, Time_Series_Analysis, Statistics
Last Updated	2026-02-08 12:00 GMT

Overview

Time series forecasting uses historical temporal observations to predict future values by decomposing a series into level, trend, and seasonal components and fitting parametric models that capture autoregressive and moving-average dynamics.

Description

Time series forecasting addresses the fundamental problem of predicting future observations based on sequentially ordered historical data. Two major families of methods dominate classical time series forecasting:

Exponential Smoothing (Holt-Winters): This method decomposes a time series into three components: level (the baseline value), trend (the rate of increase or decrease), and seasonality (recurring periodic patterns). The Holt-Winters method supports both additive seasonality (where seasonal effects are constant in magnitude) and multiplicative seasonality (where seasonal effects scale proportionally with the level). The model is parameterized by three smoothing constants (alpha, beta, gamma) that control how much weight is given to recent versus historical observations for each component. These parameters are optimized using the L-BFGS quasi-Newton method, which efficiently finds the parameter values that minimize the forecast error.

ARIMA (AutoRegressive Integrated Moving Average): ARIMA models capture temporal dependencies through three mechanisms: autoregression (AR) on past values, differencing (I) for stationarity, and moving average (MA) on past forecast errors. Seasonal ARIMA (SARIMA) extends this with seasonal counterparts parameterized by the order tuple (p, d, q)(P, D, Q, s), where p/P are AR orders, d/D are differencing orders, q/Q are MA orders, and s is the seasonal period. The model fitting uses a Kalman filter for likelihood evaluation combined with L-BFGS optimization in a batched fashion to fit multiple independent series simultaneously.

Stationarity Testing: Before fitting ARIMA models, it is necessary to determine the appropriate differencing orders (d, D). The KPSS (Kwiatkowski-Phillips-Schmidt-Shin) test is used to assess whether a series is stationary around a deterministic trend. The test null hypothesis is stationarity, so rejection indicates the need for further differencing.

Synthetic ARIMA Data: For testing and validation, synthetic time series following a known ARIMA process can be generated with specified orders, scale, noise level, and intercept.

Usage

Time series forecasting is the right choice when:

• Data is temporally ordered with regular intervals (e.g., daily sales, monthly temperatures, hourly sensor readings).
• The goal is to project future values beyond the observed range.
• Holt-Winters is preferred for series with clear trend and seasonality patterns and when interpretability of components is valued.
• ARIMA is preferred when autocorrelation structure is complex or when the practitioner needs fine control over the model order.
• Batched forecasting is needed when many independent time series (e.g., per-product, per-sensor) must be forecast simultaneously on GPU.

Theoretical Basis

Holt-Winters Additive Update Equations:

$l_t=\alpha (y_t-s_{t-m})+(1-\alpha )(l_{t-1}+b_{t-1})$

$b_t=\beta (l_t-l_{t-1})+(1-\beta )b_{t-1}$

$s_t=\gamma (y_t-l_t)+(1-\gamma )s_{t-m}$

${\hat{y}}_{t+h}=l_t+h\cdot b_t+s_{t+h-m}$

where $l_t$ is the level, $b_t$ is the trend, $s_t$ is the seasonal component, $m$ is the seasonal period, and $\alpha ,\beta ,\gamma \in [0,1]$ are the smoothing parameters.

ARIMA(p,d,q) Model:

$\varphi (B)(1-B{)}^d{y}_t=\theta (B){ϵ}_t$

where $B$ is the backshift operator, $\varphi (B)=1-{\varphi }_{1}B-\cdots -{\varphi }_p{B}^p$ is the AR polynomial, $\theta (B)=1+{\theta }_{1}B+\cdots +{\theta }_q{B}^q$ is the MA polynomial, and ${ϵ}_t\sim N(0,{\sigma }^2)$.

L-BFGS Optimization:

The smoothing parameters and ARIMA coefficients are estimated by minimizing the negative log-likelihood. L-BFGS approximates the inverse Hessian using a limited history of gradient differences, with a line search procedure controlled by the Armijo condition for step size selection. The optimization terminates when the parameter difference, error difference, or gradient norm falls below specified thresholds.

KPSS Test Statistic:

$\eta =\frac{\sum _{t=1}^T{S}_t^2}{T^2{\hat{\sigma }}^2}$

where $S_t=\sum _{i=1}^t{e}_i$ is the partial sum of regression residuals, and ${\hat{\sigma }}^2$ is a long-run variance estimator.

Related Pages

Implemented By

• Implementation:Rapidsai_Cuml_HoltWinters_C_API
• Implementation:Rapidsai_Cuml_Batched_LBFGS
• Implementation:Rapidsai_Cuml_Seasonality_Test