Implementation:Sktime Pytorch forecasting PositionalEmbedding
| Knowledge Sources | |
|---|---|
| Domains | Time_Series, Forecasting, Deep_Learning |
| Last Updated | 2026-02-08 08:00 GMT |
Overview
PositionalEmbedding provides sinusoidal positional encoding for time series sequences, enabling transformer-based models to capture temporal ordering information.
Description
The PositionalEmbedding class generates fixed (non-learnable) sinusoidal positional encodings following the approach introduced in "Attention Is All You Need." It precomputes a matrix of sine and cosine values for positions up to a configurable maximum length, using alternating sine (even indices) and cosine (odd indices) functions with exponentially increasing wavelengths. The encodings are registered as a non-gradient buffer.
Usage
Use PositionalEmbedding when building transformer-style time series models that need position-aware representations. It is used internally by EnEmbedding and can be employed in any architecture where fixed positional information must be injected into sequence embeddings.
Code Reference
Source Location
- Repository: Sktime_Pytorch_forecasting
- File: pytorch_forecasting/layers/_embeddings/_positional_embedding.py
- Lines: 1-39
Signature
class PositionalEmbedding(nn.Module):
def __init__(self, d_model, max_len=5000):
...
def forward(self, x):
...
Import
from pytorch_forecasting.layers import PositionalEmbedding
I/O Contract
Inputs
__init__ Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| d_model | int | Yes | Dimension of the model embedding space. Determines the width of each positional encoding vector. |
| max_len | int | No | Maximum length of the input sequence. Defaults to 5000. Positional encodings are precomputed for positions 0 through max_len-1. |
forward Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| x | torch.Tensor | Yes | Input tensor of shape (batch_size, seq_len, ...). Only the second dimension (seq_len) is used to determine how many positional encodings to return. |
Outputs
| Name | Type | Description |
|---|---|---|
| pe | torch.Tensor | Positional encoding tensor of shape (1, seq_len, d_model), broadcast-ready for addition to input embeddings. |
Usage Examples
import torch
from pytorch_forecasting.layers import PositionalEmbedding
# Create a positional embedding for model dimension 128
pos_embed = PositionalEmbedding(d_model=128, max_len=5000)
# Simulate an input tensor: batch=16, seq_len=96, features=128
x = torch.randn(16, 96, 128)
# Get positional encodings for the sequence length
pe = pos_embed(x)
# pe shape: (1, 96, 128)
# Add positional encodings to input
x_with_pos = x + pe
print(x_with_pos.shape) # torch.Size([16, 96, 128])