Overview
Provides utility functions for training, evaluating, visualizing, and generating predictions with LightFM hybrid recommendation models.
Description
This module contains seven utility functions for working with LightFM models. track_model_metrics trains a LightFM model epoch-by-epoch using fit_partial, recording precision@k and recall@k on both train and test interaction sets at each epoch, then formats the results into a tidy-format DataFrame and optionally generates seaborn scatter plots via model_perf_plots. compare_metric combines multiple model performance DataFrames for side-by-side comparison. similar_users and similar_items compute cosine similarity using LightFM's internal user and item representations to find the top-N nearest neighbors. prepare_test_df maps internal user/item indices back to external IDs and retrieves ratings from the interaction matrix for evaluation. prepare_all_predictions generates predictions for all user-item pairs not already in the interaction matrix by iterating over unique users and items, mapping them to internal indices, and calling the LightFM predict method with optional user and item features.
Usage
Use this module when integrating LightFM into a recommendation pipeline. Use track_model_metrics to train and monitor model performance over epochs. Use similar_users and similar_items for nearest-neighbor queries based on learned embeddings. Use prepare_test_df and prepare_all_predictions to prepare data for evaluation with the recommenders evaluation framework.
Code Reference
Source Location
Signature
def model_perf_plots(df)
def compare_metric(df_list, metric="prec", stage="test")
def track_model_metrics(
model,
train_interactions,
test_interactions,
k=10,
no_epochs=100,
no_threads=8,
show_plot=True,
**kwargs
)
def similar_users(user_id, user_features, model, N=10)
def similar_items(item_id, item_features, model, N=10)
def prepare_test_df(test_idx, uids, iids, uid_map, iid_map, weights)
def prepare_all_predictions(
data,
uid_map,
iid_map,
interactions,
model,
num_threads,
user_features=None,
item_features=None,
)
Import
from recommenders.models.lightfm.lightfm_utils import (
track_model_metrics,
similar_users,
similar_items,
prepare_test_df,
prepare_all_predictions,
compare_metric,
model_perf_plots,
)
I/O Contract
Inputs
track_model_metrics
| Name |
Type |
Required |
Description
|
| model |
LightFM |
Yes |
A LightFM model instance to be trained
|
| train_interactions |
scipy.sparse.coo_matrix |
Yes |
Training user-item interaction matrix
|
| test_interactions |
scipy.sparse.coo_matrix |
Yes |
Test user-item interaction matrix
|
| k |
int |
No |
Number of recommendations for precision/recall@k (default 10)
|
| no_epochs |
int |
No |
Number of training epochs (default 100)
|
| no_threads |
int |
No |
Number of parallel threads (default 8)
|
| show_plot |
bool |
No |
Whether to display performance plots (default True)
|
| **kwargs |
dict |
No |
Additional keyword arguments passed to fit_partial and evaluation functions
|
similar_users
| Name |
Type |
Required |
Description
|
| user_id |
int |
Yes |
ID of the reference user
|
| user_features |
scipy.sparse.csr_matrix |
Yes |
User feature matrix
|
| model |
LightFM |
Yes |
Fitted LightFM model
|
| N |
int |
No |
Number of top similar users to return (default 10)
|
similar_items
| Name |
Type |
Required |
Description
|
| item_id |
int |
Yes |
ID of the reference item
|
| item_features |
scipy.sparse.csr_matrix |
Yes |
Item feature matrix
|
| model |
LightFM |
Yes |
Fitted LightFM model
|
| N |
int |
No |
Number of top similar items to return (default 10)
|
prepare_all_predictions
| Name |
Type |
Required |
Description
|
| data |
pd.DataFrame |
Yes |
DataFrame of all users, items, and ratings
|
| uid_map |
dict |
Yes |
Mapping from external user IDs to internal indices
|
| iid_map |
dict |
Yes |
Mapping from external item IDs to internal indices
|
| interactions |
scipy.sparse.coo_matrix |
Yes |
User-item interaction matrix
|
| model |
LightFM |
Yes |
Fitted LightFM model
|
| num_threads |
int |
Yes |
Number of parallel computation threads
|
| user_features |
scipy.sparse.csr_matrix |
No |
User feature weights (default None)
|
| item_features |
scipy.sparse.csr_matrix |
No |
Item feature weights (default None)
|
Outputs
track_model_metrics
| Name |
Type |
Description
|
| fitting_metrics |
pd.DataFrame |
Tidy-format DataFrame with columns: epoch, value, stage (train/test), metric (Precision/Recall)
|
| model |
LightFM |
The trained LightFM model after all epochs
|
similar_users
| Name |
Type |
Description
|
| return |
pd.DataFrame |
Top N most similar users with columns: userID, score
|
similar_items
| Name |
Type |
Description
|
| return |
pd.DataFrame |
Top N most similar items with columns: itemID, score
|
prepare_all_predictions
| Name |
Type |
Description
|
| return |
pd.DataFrame |
Predictions for all unseen user-item pairs with columns: userID, itemID, prediction
|
Usage Examples
Basic Usage
from lightfm import LightFM
from recommenders.models.lightfm.lightfm_utils import (
track_model_metrics,
similar_users,
similar_items,
prepare_all_predictions,
)
# Initialize and train a LightFM model with metric tracking
model = LightFM(loss="warp", no_components=30)
metrics_df, trained_model = track_model_metrics(
model=model,
train_interactions=train_interactions,
test_interactions=test_interactions,
k=10,
no_epochs=50,
no_threads=4,
show_plot=True,
)
# Find top 10 similar users to user 0
similar_user_df = similar_users(
user_id=0,
user_features=user_features,
model=trained_model,
N=10,
)
# Find top 10 similar items to item 5
similar_item_df = similar_items(
item_id=5,
item_features=item_features,
model=trained_model,
N=10,
)
# Generate all predictions for evaluation
all_preds = prepare_all_predictions(
data=ratings_df,
uid_map=uid_map,
iid_map=iid_map,
interactions=interactions,
model=trained_model,
num_threads=4,
user_features=user_features,
item_features=item_features,
)
Related Pages
