Principle:Pola rs Polars Time Series Resampling
| Knowledge Sources | |
|---|---|
| Domains | Data Engineering, Time Series |
| Last Updated | 2026-02-09 10:00 GMT |
Overview
Changing the frequency of time series data by upsampling (increasing frequency) with interpolation or fill strategies to handle the introduced null values.
Description
Time series resampling adjusts the temporal resolution of a dataset. Upsampling increases the frequency (e.g., from 30-minute intervals to 15-minute intervals) by inserting new rows at the desired cadence. The newly inserted rows contain null values in all non-index columns, since no observations exist at those intermediate time points. These nulls must then be filled using an appropriate strategy.
Polars provides a two-step workflow for upsampling:
upsample-- Inserts new rows at the specified frequency, filling the temporal index column with evenly-spaced values. All other columns receive null values at the new time points.- Null-filling strategy -- The nulls introduced by upsampling are filled using one of several approaches:
- Forward fill (
fill_null(strategy="forward")) -- Propagates the last known value forward to fill gaps. This assumes the quantity remains constant until the next observation (step function interpolation). - Backward fill (
fill_null(strategy="backward")) -- Propagates the next known value backward. - Interpolation (
interpolate()) -- Estimates intermediate values using linear interpolation between adjacent known values. This assumes a linear trend between observations. - Constant fill (
fill_null(strategy="zero"),fill_null(strategy="one")) -- Fills with a fixed value.
- Forward fill (
The choice of fill strategy depends on the domain semantics: forward fill is appropriate for quantities that change at discrete events (e.g., stock prices between trades), while interpolation is appropriate for quantities that change continuously (e.g., temperature readings).
Usage
Use this principle whenever you need to:
- Increase the temporal resolution of a time series for alignment with higher-frequency data.
- Align multiple time series with different sampling frequencies to a common grid.
- Fill temporal gaps in irregularly-sampled data.
- Prepare uniformly-spaced data for models that require regular time steps.
Theoretical Basis
Upsampling inserts rows at a higher frequency, creating gaps that must be filled. This operation originates from signal processing resampling theory and interpolation theory.
The upsampling operation can be formalized as:
upsample(D, every) = D' where:
D'.time = {t_0, t_0 + every, t_0 + 2*every, ..., t_n}
D'.value[i] = D.value[i] if t_i exists in D.time
D'.value[i] = null otherwise
The null-filling strategies correspond to different interpolation methods:
| Strategy | Mathematical Basis | Assumption |
|---|---|---|
| Forward fill | Zero-order hold: f(t) = f(t_prev) |
Value is constant until next observation |
| Backward fill | Reverse zero-order hold: f(t) = f(t_next) |
Value matches the next observation |
| Linear interpolation | f(t) = f(t_prev) + (t - t_prev) / (t_next - t_prev) * (f(t_next) - f(t_prev)) |
Value changes linearly between observations |
| Constant fill | f(t) = c |
Missing values default to a known constant (0, 1, etc.) |
Linear interpolation minimizes the maximum error under the assumption of bounded second derivatives (i.e., when the true signal is approximately linear between sample points). For signals with higher-order smoothness, spline or polynomial interpolation may be more appropriate, though Polars' built-in interpolate() uses linear interpolation.
The dual operation -- downsampling (reducing frequency) -- is typically achieved through group_by_dynamic with an appropriate aggregation function (mean, sum, last, etc.).