Implementation:Sktime Pytorch forecasting Embedding Cat Variables
| Knowledge Sources | |
|---|---|
| Domains | Time_Series, Forecasting, Deep_Learning |
| Last Updated | 2026-02-08 08:00 GMT |
Overview
embedding_cat_variables is a sub-network module that embeds categorical variables and augments them with three positional features (sequence position, future position, and future indicator) for time series forecasting models.
Description
The embedding_cat_variables class creates learned embeddings for each categorical variable and automatically generates three additional positional variables during the forward pass: a sequence position index, a future-step position indicator, and a binary past/future flag. Each variable (both user-provided categorical features and the generated positional features) is independently embedded into a shared hidden dimension using separate nn.Embedding layers, producing a rich representation suitable for attention-based architectures.
Usage
Use embedding_cat_variables when your time series model needs to incorporate categorical covariates (e.g., day of week, product ID) alongside temporal positional information. It is particularly suited for encoder-decoder architectures where distinguishing past context from future prediction steps is important, such as in the DSiPTS model family.
Code Reference
Source Location
- Repository: Sktime_Pytorch_forecasting
- File: pytorch_forecasting/layers/_embeddings/_sub_nn.py
- Lines: 1-99
Signature
class embedding_cat_variables(nn.Module):
def __init__(self, seq_len: int, lag: int, d_model: int, emb_dims: list, device):
...
def forward(self, x: torch.Tensor | int, device: torch.device) -> torch.Tensor:
...
Import
from pytorch_forecasting.layers import embedding_cat_variables
I/O Contract
Inputs
__init__ Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| seq_len | int | Yes | Total length of the sequence (sum of past and future steps). |
| lag | int | Yes | Number of future steps to be predicted. |
| d_model | int | Yes | Dimension of all variables after they are embedded. |
| emb_dims | list | Yes | List of dictionary sizes for embedding, one dimension per categorical variable. |
| device | torch.device | Yes | Device on which tensors are allocated. |
forward Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| x | torch.Tensor or int | Yes | Either a tensor of shape (batch_size, seq_len, num_vars) containing categorical variable indices, or an integer representing the batch size when no categorical variables are provided. |
| device | torch.device | Yes | Device on which to place the generated positional tensors. |
Outputs
| Name | Type | Description |
|---|---|---|
| cat_n_embd | torch.Tensor | Embedded tensor of shape (batch_size, seq_len, num_vars + 3, d_model), where the 3 additional variables are pos_seq, pos_fut, and is_fut. |
Usage Examples
import torch
from pytorch_forecasting.layers import embedding_cat_variables
device = torch.device("cpu")
# Configuration: 96 total steps, 24 future steps, embedding dim 32
# Two categorical variables with dictionary sizes 10 and 5
emb_layer = embedding_cat_variables(
seq_len=96, lag=24, d_model=32, emb_dims=[10, 5], device=device
)
# Input: batch of 8, 96 time steps, 2 categorical variables
x = torch.randint(0, 5, (8, 96, 2))
# Forward pass
output = emb_layer(x, device=device)
# output shape: (8, 96, 5, 32) -> 2 cat vars + 3 positional vars, each embedded to 32
print(output.shape)
# When no categorical variables are provided, pass batch size as int
output_no_cat = emb_layer(8, device=device)
# output shape: (8, 96, 3, 32) -> only 3 positional vars
print(output_no_cat.shape)