Implementation:Sktime Pytorch forecasting DataEmbedding Inverted
| Knowledge Sources | |
|---|---|
| Domains | Time_Series, Forecasting, Deep_Learning |
| Last Updated | 2026-02-08 08:00 GMT |
Overview
A channel-independent inverted data embedding layer that projects time series data from the temporal dimension into the model dimension.
Description
DataEmbedding_inverted is an nn.Module that implements an inverted embedding strategy for time series data. Unlike standard embeddings that project along the feature dimension, this layer first permutes the input tensor from (Batch, Time, Variate) to (Batch, Variate, Time), then applies a linear projection from the temporal dimension (c_in, representing the number of time steps) to the model dimension (d_model). When temporal marks (x_mark) are provided, they are concatenated along the variate dimension before projection.
This inverted approach treats each variate independently and is particularly effective for channel-independent forecasting models where cross-variate dependencies are not explicitly modeled.
Usage
Use this embedding layer in iTransformer-style or other channel-independent forecasting architectures where the model dimension should represent compressed temporal information rather than feature embeddings. It is placed at the input stage of the encoder to transform raw time series into the format expected by subsequent attention or feed-forward layers.
Code Reference
Source Location
- Repository: Sktime_Pytorch_forecasting
- File: pytorch_forecasting/layers/_embeddings/_data_embedding.py
- Lines: 1-38
Signature
class DataEmbedding_inverted(nn.Module):
def __init__(self, c_in, d_model, dropout=0.1)
def forward(self, x, x_mark)
Import
from pytorch_forecasting.layers._embeddings._data_embedding import DataEmbedding_inverted
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| c_in | int | Yes | Number of input time steps (temporal dimension size) |
| d_model | int | Yes | Target model dimension for the linear projection |
| dropout | float | No | Dropout rate applied after embedding (default: 0.1) |
| x | torch.Tensor | Yes | Input time series tensor of shape (Batch, Time, Variate) |
| x_mark | torch.Tensor or None | Yes | Optional temporal mark tensor of shape (Batch, Time, Mark_Features). If None, only x is embedded; if provided, concatenated along variate dimension before projection |
Outputs
| Name | Type | Description |
|---|---|---|
| output | torch.Tensor | Embedded tensor of shape (Batch, Variate, d_model) when x_mark is None, or (Batch, Variate + Mark_Features, d_model) when x_mark is provided |
Usage Examples
import torch
from pytorch_forecasting.layers._embeddings._data_embedding import DataEmbedding_inverted
# Create an inverted embedding for 96 time steps projected to 512 dimensions
embedding = DataEmbedding_inverted(c_in=96, d_model=512, dropout=0.1)
# Embed without temporal marks
x = torch.randn(32, 96, 7) # (batch, time, variates)
output = embedding(x, x_mark=None)
print(output.shape) # torch.Size([32, 7, 512])
# Embed with temporal marks
x_mark = torch.randn(32, 96, 4) # (batch, time, mark_features)
output = embedding(x, x_mark=x_mark)
print(output.shape) # torch.Size([32, 11, 512]) # 7 variates + 4 mark features