Principle:Recommenders team Recommenders Neural Collaborative Filtering
| Knowledge Sources | |
|---|---|
| Domains | Recommender Systems, Deep Learning, Collaborative Filtering |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
Neural Collaborative Filtering (NCF) replaces the fixed inner product of traditional matrix factorization with a learned neural network function, enabling non-linear modeling of user-item interactions.
Description
Classical matrix factorization models predict a user's preference for an item by computing the dot product of their respective latent factor vectors. While effective, the dot product is a fixed linear function that limits the model's ability to capture complex, non-linear interaction patterns between user and item embeddings.
Neural Collaborative Filtering (He et al., WWW 2017) proposes replacing the inner product with a multi-layer neural network that can learn an arbitrary function from user and item embeddings to a predicted score. The key equation is:
y_hat(u, i) = f(p_u, q_i | Theta)
where p_u is the user embedding, q_i is the item embedding, f is a neural network, and Theta represents all learnable parameters.
The paper introduces three model variants:
GMF (Generalized Matrix Factorization): Computes the element-wise product of user and item embeddings, followed by a single output layer. This generalizes the dot product by allowing each latent dimension to have a different learned weight.
MLP (Multi-Layer Perceptron): Concatenates user and item embeddings and feeds them through multiple fully-connected layers with non-linear activations (ReLU). This allows the model to learn arbitrary non-linear functions of the combined embedding space.
NeuMF (Neural Matrix Factorization): Combines GMF and MLP by concatenating their respective last hidden layers before the final prediction layer. This fuses the linear interaction modeling of GMF with the non-linear capacity of MLP.
The model is trained with binary cross-entropy loss over positive (observed) and negative (sampled unobserved) user-item pairs, treating recommendation as a binary classification problem on implicit feedback.
Usage
Use NCF when you need a neural approach to collaborative filtering on implicit feedback data. Choose GMF for a lightweight baseline that generalizes dot-product matrix factorization. Choose MLP when you want to model complex non-linear user-item interactions. Choose NeuMF (the default) for the strongest variant that combines both approaches. NeuMF can optionally be initialized with pre-trained GMF and MLP weights for better convergence.
Theoretical Basis
GMF Layer
phi_GMF(p_u, q_i) = p_u (*) q_i # element-wise product
y_hat = sigmoid(h^T * phi_GMF) # output layer
where (*) denotes element-wise (Hadamard) product and h is a learned weight vector.
MLP Layers
z_0 = [p_u ; q_i] # concatenation
z_1 = ReLU(W_1 * z_0 + b_1) # hidden layer 1
z_2 = ReLU(W_2 * z_1 + b_2) # hidden layer 2
...
y_hat = sigmoid(h^T * z_L) # output layer
The layer sizes form a tower pattern (e.g., [16, 8, 4]), progressively compressing the representation.
NeuMF Fusion
phi_GMF = p_u^GMF (*) q_i^GMF
phi_MLP = z_L # last MLP hidden layer
y_hat = sigmoid(h^T * [phi_GMF ; phi_MLP]) # concatenate and predict
Note that GMF and MLP use separate embedding layers, allowing each pathway to learn different latent representations.
Loss Function
Binary cross-entropy over positive set R+ and negative-sampled set R-:
L = - sum_{(u,i) in R+} log(y_hat(u,i))
- sum_{(u,j) in R-} log(1 - y_hat(u,j))
Pre-training Strategy
NeuMF can be initialized by first training GMF and MLP separately, then loading their weights into the corresponding NeuMF sub-networks. The final output layer is initialized as the concatenation of the pre-trained GMF and MLP output weights, scaled by a factor alpha.