Principle:Fastai Fastbook Neural Collaborative Filtering
| Knowledge Sources | |
|---|---|
| Domains | Recommender Systems, Deep Learning, Collaborative Filtering |
| Last Updated | 2026-02-09 17:00 GMT |
Overview
Neural collaborative filtering replaces the fixed dot-product interaction function with a learned nonlinear function implemented as a multi-layer neural network that takes concatenated user and item embeddings as input and outputs a predicted rating.
Description
The dot-product approach described in the latent factor model is a bilinear interaction: it can only capture linear relationships between user and item latent dimensions. Neural collaborative filtering (NCF), introduced by He et al. (2017), generalizes this by passing the concatenation of user and item embedding vectors through one or more fully connected hidden layers with nonlinear activations (ReLU). This allows the model to learn arbitrary nonlinear interaction patterns between users and items.
The key architectural difference from the dot-product model:
- Dot product: The user and item embeddings must have the same dimensionality k, and interaction is computed as
sum(p_u * q_i). - Neural network: The user and item embeddings can have different dimensionalities. They are concatenated into a single vector of length
k_u + k_i, then fed through a sequence ofLinear -> ReLU -> Linear -> ... -> Linear(1)layers.
Because the concatenated embeddings pass through a general-purpose function approximator, the neural approach can in theory capture more complex patterns. However, in practice on pure collaborative filtering benchmarks, it often performs comparably to or slightly worse than well-tuned dot-product models. The real advantage of the neural approach is its extensibility: because the architecture is based on a standard tabular model, it can seamlessly incorporate additional features (user demographics, item metadata, timestamps, etc.) as continuous or categorical inputs alongside the embeddings.
Usage
Use the neural collaborative filtering approach when:
- You want to incorporate side information (user metadata, item features) beyond just the user-item interaction.
- You have reason to believe nonlinear interactions between latent factors are important.
- You plan to extend the model into a full tabular recommendation system.
For pure user-item rating prediction without side features, start with the dot-product model and switch to neural only if you need the additional flexibility.
Theoretical Basis
Architecture
The neural collaborative filtering forward pass can be described in pseudocode:
Input: (user_idx, item_idx)
1. e_u = UserEmbedding(user_idx) # shape: (batch, k_u)
2. e_i = ItemEmbedding(item_idx) # shape: (batch, k_i)
3. h_0 = concatenate(e_u, e_i) # shape: (batch, k_u + k_i)
4. For each hidden layer l = 1, ..., L:
h_l = ReLU( W_l * h_{l-1} + b_l )
5. output = W_{L+1} * h_L + b_{L+1} # shape: (batch, 1)
6. prediction = sigmoid_range(output, low, high)
For the fastbook MovieLens example with layers=[100, 50]:
- Embedding sizes are determined by the
get_emb_szheuristic: (944, 74) for users and (1635, 101) for items h_0has 74 + 101 = 175 features- Hidden layer 1: Linear(175, 100) followed by ReLU
- Hidden layer 2: Linear(100, 50) followed by ReLU
- Output layer: Linear(50, 1)
Relationship to Tabular Models
In fastai, EmbeddingNN is a thin subclass of TabularModel with n_cont=0 (no continuous features) and out_sz=1 (single rating output). This means that neural collaborative filtering is literally a tabular model that happens to have only two categorical inputs (user and item). This architectural choice makes it trivial to add continuous features or additional categorical features later.
Embedding Size Heuristic
The get_emb_sz function in fastai computes recommended embedding dimensionality using a rule of thumb:
emb_dim = min(600, round(1.6 * n_categories^0.56))
This produces larger embeddings for categorical variables with many unique values, while capping at 600 to prevent excessive memory usage.
Training and Regularization
Like the dot-product model, the neural model is trained with MSE loss and weight decay. However, because the neural model has more parameters (hidden layer weights in addition to embeddings), appropriate weight decay is important to prevent overfitting. The fastbook example uses wd=0.1.