Implementation:Recommenders team Recommenders SUM Model

Knowledge Sources	Recommenders
Domains	Recommendation Systems, Deep Learning, Sequential Recommendation
Last Updated	2026-02-10 00:00 GMT

Overview

Implements the SUM (Sequential User Matrix) model for multi-interest sequential recommendation, representing users with multiple memory channels that capture diverse preferences simultaneously.

Description

The SUMModel class extends SequentialBaseModel and implements _build_seq_graph with a multi-slot memory architecture.

The _create_sumcell factory method initializes a SUM cell (either SUMCell or SUMV2Cell based on the hparams.cell configuration) with parameters derived from the model configuration: input size (hidden_size * slots + input_embedding_dim), number of slots, attention size, and input embedding dimension.

The _build_sum method processes the concatenated item and category history embeddings through the SUM cell using dynamic_rnn with sequence length masking and a zero initial state. The SUM cell maintains multiple memory slots, producing a state tensor of shape (batch_size, slots * hidden_size). After RNN processing, the state is trimmed to retain only the memory slot portion.

The _attention_query_by_state method handles the fusion of multiple memory channels conditioned on the target item. When slots > 1, it reshapes the flat state into (batch_size, slots, hidden_size), then uses a learned query_att_W matrix to compute attention weights via softmax(memory @ W @ query). The weighted sum of memory slots produces the final user representation. When slots == 1, the state passes through directly.

The final model_output is the concatenation of this merged user representation with the target_item_embedding.

The model is based on the work by Lian et al. (2021) on multi-interest-aware user modeling for large-scale sequential recommendations.

Usage

Use this model when users have diverse or multi-faceted interests that a single user vector would fail to capture. The multi-slot memory architecture is particularly beneficial for large-scale settings where users interact with items across many categories or topics.

Code Reference

Source Location

• Repository: Recommenders
• File: recommenders/models/deeprec/models/sequential/sum.py
• Lines: 1-154

Signature

class SUMModel(SequentialBaseModel):
    def _build_seq_graph(self)

    def _attention_query_by_state(self, seq_output, query)

    def _create_sumcell(self)

    def _build_sum(self, cell)

Import

from recommenders.models.deeprec.models.sequential.sum import SUMModel

I/O Contract

Inputs

Name	Type	Required	Description
self.item_history_embedding	tf.Tensor	Yes	Embedding tensor of the user's item interaction history
self.cate_history_embedding	tf.Tensor	Yes	Embedding tensor of the corresponding category history
self.target_item_embedding	tf.Tensor	Yes	Embedding tensor of the target item being scored
self.iterator.mask	tf.Tensor	Yes	Binary mask tensor indicating valid sequence positions
hparams.hidden_size	int	Yes	Hidden size for each memory slot
hparams.slots	int	Yes	Number of memory channels/slots for multi-interest modeling
hparams.attention_size	int	Yes	Dimensionality of the attention layer within the SUM cell
hparams.cell	str	Yes	Type of SUM cell to use: "SUM" or "SUMV2"
seq_output (_attention_query_by_state)	tf.Tensor	Yes	Flattened SUM memory states of shape (batch_size, slots * hidden_size)
query (_attention_query_by_state)	tf.Tensor	Yes	Target item embedding used to attend over memory slots
cell (_build_sum)	object	Yes	An initialized SUM cell (SUMCell or SUMV2Cell)

Outputs

Name	Type	Description
return (_build_seq_graph)	tf.Tensor	Concatenation of the attention-merged user representation and target item embedding
return (_attention_query_by_state)	tuple(tf.Tensor, tf.Tensor)	Merged user representation and attention weights of each memory channel
return (_create_sumcell)	object	An initialized SUM cell configured with model hyperparameters
return (_build_sum)	tf.Tensor	Flattened representation of user memory states of shape (batch_size, slots * hidden_size)

Usage Examples

Basic Usage

from recommenders.models.deeprec.models.sequential.sum import SUMModel
from recommenders.models.deeprec.deeprec_utils import prepare_hparams

# Prepare hyperparameters from YAML config
hparams = prepare_hparams(
    "recommenders/models/deeprec/config/sum.yaml",
    hidden_size=64,
    slots=4,           # number of memory channels
    attention_size=40,
    cell="SUM",         # or "SUMV2"
)

# Create and train the SUM model
model = SUMModel(hparams, iterator_creator)
model.fit(train_file, valid_file)

# Evaluate the model
eval_results = model.run_eval(test_file)