Implementation:Recommenders team Recommenders BaseModel Run Fast Eval

Knowledge Sources	Recommenders
Domains	News Recommendation, Inference, Embedding
Last Updated	2026-02-10 00:00 GMT

Overview

Concrete tool for performing fast news recommendation inference by pre-computing news and user embedding vectors and scoring candidates via numpy dot products.

Description

BaseModel.run_fast_eval implements the fast evaluation strategy for neural news recommendation models. It performs three sequential operations:

News vector computation — Calls self.run_news(news_filename), which iterates over all news articles using the test iterator's load_news_from_file method. Each article is encoded by self.newsencoder.predict_on_batch. Returns a dictionary mapping news IDs to their embedding vectors.
User vector computation — Calls self.run_user(news_filename, behaviors_file), which iterates over all user impressions using the test iterator's load_user_from_file method. Each user's click history is encoded by self.userencoder.predict_on_batch. Returns a dictionary mapping impression indices to user embedding vectors.
Fast scoring loop — Iterates over impressions from self.test_iterator.load_impression_from_file(behaviors_file). For each impression:
- Stacks the pre-computed news vectors for all candidate articles into a matrix.
- Retrieves the pre-computed user vector for this impression.
- Computes prediction scores via np.dot(news_matrix, user_vector).
- Appends the impression index, labels, and predictions to output lists.

The pre-computed vectors are also stored as self.news_vecs and self.user_vecs for potential downstream use.

Usage

run_fast_eval is called automatically by run_eval when support_quick_scoring=True. It can also be called directly when you need access to the raw grouped predictions (e.g., for generating submission files or custom metric computation).

Code Reference

Source Location

Repository: recommenders-team/recommenders
File: recommenders/models/newsrec/models/base_model.py (lines 404-429)

Signature

def run_fast_eval(
    self, news_filename: str, behaviors_file: str
) -> tuple[list, list, list]:
    """Fast evaluation using pre-computed news and user embeddings.

    Args:
        news_filename (str): Path to the news metadata file (news.tsv).
        behaviors_file (str): Path to the user behaviors file (behaviors.tsv).

    Returns:
        tuple: (group_impr_indexes, group_labels, group_preds)
            - group_impr_indexes (list): Impression indices.
            - group_labels (list[list]): Per-impression ground-truth labels.
            - group_preds (list[list]): Per-impression prediction scores.
    """

Import

# Accessed via the NRMSModel class (inherits from BaseModel)
from recommenders.models.newsrec.models.nrms import NRMSModel
from recommenders.models.newsrec.io.mind_iterator import MINDIterator

model = NRMSModel(hparams, MINDIterator, seed=42)
# model.run_fast_eval(...) is inherited from BaseModel

I/O Contract

Parameter	Type	Description
`news_filename`	`str`	Path to the news.tsv file containing news article metadata
`behaviors_file`	`str`	Path to the behaviors.tsv file containing user impression logs

Return	Type	Description
`group_impr_indexes`	`list`	List of impression indices, one per impression
`group_labels`	`list[list]`	Per-impression lists of ground-truth click labels (0 or 1)
`group_preds`	`list[numpy.ndarray]`	Per-impression arrays of predicted scores (dot product values)

Side Effects

Effect	Description
`self.news_vecs`	Dictionary mapping news IDs to pre-computed news embedding vectors
`self.user_vecs`	Dictionary mapping impression indices to pre-computed user embedding vectors

Usage Examples

import os
import numpy as np

valid_news_file = os.path.join(data_path, "valid", "news.tsv")
valid_behaviors_file = os.path.join(data_path, "valid", "behaviors.tsv")

# Run fast evaluation to get raw predictions
group_impr_indexes, group_labels, group_preds = model.run_fast_eval(
    valid_news_file, valid_behaviors_file
)

# Inspect predictions for the first impression
print(f"Impression: {group_impr_indexes[0]}")
print(f"Labels:     {group_labels[0]}")
print(f"Preds:      {group_preds[0]}")

# Access pre-computed embeddings for downstream use
news_embeddings = model.news_vecs    # dict: news_id -> np.ndarray
user_embeddings = model.user_vecs    # dict: impr_index -> np.ndarray

# Generate a submission file
with open("prediction.txt", "w") as f:
    for impr_index, preds in zip(group_impr_indexes, group_preds):
        ranked = np.argsort(preds)[::-1] + 1  # 1-indexed ranks
        rank_str = ",".join([str(r) for r in ranked])
        f.write(f"{impr_index} [{rank_str}]\n")

Dependencies

tensorflow — For running the news encoder and user encoder via predict_on_batch
numpy — For np.dot scoring and np.stack for assembling news matrices
tqdm — For progress bars during news and user encoding phases

Related Pages

Implements Principle

Principle:Recommenders_team_Recommenders_News_Prediction_Export

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment