Implementation:Tensorflow Tfjs ConvLSTM2D Layer

Knowledge Sources	Tensorflow_Tfjs TensorFlow.js API
Domains	Deep_Learning, Layers_API
Last Updated	2026-02-10 06:00 GMT

Overview

The convolutional_recurrent.ts module implements the ConvLSTM2D layer, which combines convolutional operations with LSTM recurrence. This is useful for spatiotemporal sequence modeling tasks such as video prediction, weather forecasting, and other problems involving sequences of images or spatial data.

The architecture consists of four classes:

• ConvRNN2DCell (abstract): Base class for convolutional RNN cells, defining shared properties (filters, kernelSize, strides, padding, dataFormat, dilationRate).
• ConvLSTM2DCell: Concrete cell implementing the LSTM gates (input, forget, cell, output) using 2D convolutions instead of dense matrix multiplications.
• ConvRNN2D (abstract): Base wrapper class that extends RNN for convolutional RNN layers, handling 5D input (batch, time, height, width, channels).
• ConvLSTM2D: The user-facing layer that creates a ConvLSTM2DCell and passes it to ConvRNN2D.

Code Reference

Source Location

tfjs-layers/src/layers/convolutional_recurrent.ts (View on GitHub)

Key Signatures

abstract class ConvRNN2DCell extends RNNCell {
  readonly filters: number;
  readonly kernelSize: number[];
  readonly strides: number[];
  readonly padding: PaddingMode;
  readonly dataFormat: DataFormat;
  readonly dilationRate: number[];
}

export class ConvLSTM2DCell extends LSTMCell {
  static className = 'ConvLSTM2DCell';
  constructor(args: ConvLSTM2DCellArgs);
  build(inputShape: Shape | Shape[]): void;
  call(inputs: Tensor[], kwargs: Kwargs): Tensor[];
  getConfig(): serialization.ConfigDict;
  inputConv(x: Tensor, w: Tensor, b?: Tensor, padding?: PaddingMode): Tensor;
  recurrentConv(x: Tensor, w: Tensor): Tensor;
}

class ConvRNN2D extends RNN {
  static className = 'ConvRNN2D';
  constructor(args: ConvRNN2DLayerArgs);
  call(inputs: Tensor | Tensor[], kwargs: Kwargs): Tensor | Tensor[];
  computeOutputShape(inputShape: Shape): Shape | Shape[];
  getInitialState(inputs: Tensor): Tensor[];
}

export class ConvLSTM2D extends ConvRNN2D {
  static className = 'ConvLSTM2D';
  constructor(args: ConvLSTM2DArgs);
  static fromConfig<T>(cls, config): T;
}

Import

import { ConvLSTM2D, ConvLSTM2DCell } from './layers/convolutional_recurrent';

I/O Contract

Input

• ConvLSTM2D: Rank-5 tensor of shape [batch, timesteps, height, width, channels] (channelsLast). Each timestep is a 2D spatial frame.
• ConvLSTM2DCell: Called internally with [input, hPrev, cPrev] where input is a single frame and hPrev/cPrev are the hidden and cell states from the previous timestep.

Output

• If returnSequences=true: Rank-5 tensor [batch, timesteps, outHeight, outWidth, filters].
• If returnSequences=false (default): Rank-4 tensor [batch, outHeight, outWidth, filters] (last timestep only).
• If returnState=true: Additional state tensors [h, c] are returned, each of shape [batch, outHeight, outWidth, filters].

Cell Computation

The ConvLSTM2DCell.call() method implements the standard LSTM equations with convolutions:

i = recurrentActivation(inputConv(x, kernelI) + recurrentConv(hPrev, recKernelI))
f = recurrentActivation(inputConv(x, kernelF) + recurrentConv(hPrev, recKernelF))
c = f * cPrev + i * activation(inputConv(x, kernelC) + recurrentConv(hPrev, recKernelC))
o = recurrentActivation(inputConv(x, kernelO) + recurrentConv(hPrev, recKernelO))
h = o * activation(c)

Constraints

• Unrolling is not supported for convolutional RNNs.
• Stacking convolutional cells is not supported.
• Constants are not supported in ConvRNN2D.

Usage Examples

import * as tf from '@tensorflow/tfjs';

// ConvLSTM2D for video prediction
const model = tf.sequential();
model.add(tf.layers.convLstm2d({
  inputShape: [10, 64, 64, 3],  // 10 frames of 64x64 RGB
  filters: 32,
  kernelSize: 3,
  padding: 'same',
  returnSequences: true
}));
model.add(tf.layers.convLstm2d({
  filters: 16,
  kernelSize: 3,
  padding: 'same',
  returnSequences: false
}));

// Output shape: [batch, 64, 64, 16]
const input = tf.randomNormal([2, 10, 64, 64, 3]);
const output = model.predict(input);

Related Pages

• Tensorflow_Tfjs_Recurrent_Layers - Base RNN, LSTMCell, and rnn() function that ConvLSTM2D extends
• Tensorflow_Tfjs_Convolutional_Layers - Standard convolutional layers and preprocessing utilities
• Tensorflow_Tfjs_Backend_Operations - K.biasAdd used in the cell's input convolution
• Tensorflow_Tfjs_Exports_Layers - Public API factory function tf.layers.convLstm2d()