Implementation:Evidentlyai Evidently Recsys Preset

Knowledge Sources	Evidentlyai_Evidently
Domains	Presets, Recommendation Systems
Last Updated	2026-02-14 12:00 GMT

Overview

Provides the RecsysPreset metric container that assembles a comprehensive set of recommendation system evaluation metrics including ranking quality, diversity, novelty, bias, and personalization.

Description

The Recsys Preset module defines the RecsysPreset class, a MetricContainer that generates a full suite of recommendation system metrics based on the provided configuration. The preset dynamically adjusts the set of generated metrics based on the availability of training data and feature columns.

Core metrics (always included):

• PrecisionTopK -- Precision at rank K
• RecallTopK -- Recall at rank K
• FBetaTopK -- F-beta score at rank K
• MAP -- Mean Average Precision at K
• NDCG -- Normalized Discounted Cumulative Gain at K
• MRR -- Mean Reciprocal Rank at K
• HitRate -- Hit rate at K
• ScoreDistribution -- Distribution of recommendation scores
• RecCasesTable -- Table of recommendation examples
• Personalization -- Personalization score at K

Training-data-dependent metrics (included when current_train_data is present in context.additional_data):

• PopularityBiasMetric -- Average Recommendation Popularity
• Novelty -- Novelty score at K

Item-feature-dependent metrics (included when item_features is provided):

• Diversity -- Diversity based on item features
• Serendipity -- Serendipity based on item features (also requires training data)

Bias metrics (included when training data is present):

• ItemBias -- Per-column item bias with both "default" and "train" distributions
• UserBias -- Per-column user bias with both "default" and "train" distributions

All TopK metrics support configurable k, min_rel_score, no_feedback_users, and ranking_name parameters.

Usage

Use RecsysPreset when you need a comprehensive evaluation of a recommendation system. Configure it with the top-K cutoff and optional parameters for relevance scoring, bias analysis, and diversity evaluation. Include it in an Evidently report along with the appropriate data definition for recommendation data.

Code Reference

Source Location

• Repository: Evidentlyai_Evidently
• File: src/evidently/presets/recsys.py

Signature

class RecsysPreset(MetricContainer):
    k: int
    min_rel_score: Optional[int] = None
    no_feedback_users: bool = False
    ranking_name: str = "default"
    beta: Optional[float] = 1.0
    normalize_arp: bool = False
    user_ids: Optional[List[str]] = None
    display_features: Optional[List[str]] = None
    item_features: Optional[List[str]] = None
    user_bias_columns: Optional[List[str]] = None
    item_bias_columns: Optional[List[str]] = None

    def __init__(self, k: int, ..., include_tests: bool = True):
        ...
    def generate_metrics(self, context: "Context") -> Sequence[MetricOrContainer]:
        ...

Import

from evidently.presets.recsys import RecsysPreset

I/O Contract

Inputs

Name	Type	Required	Description
k	int	Yes	Top-K cutoff for ranking metrics
min_rel_score	Optional[int]	No	Minimum relevance score threshold for binary relevance metrics
no_feedback_users	bool	No	Whether to include users with no feedback in metric calculations (default: False)
ranking_name	str	No	Name of the ranking to evaluate (default: "default")
beta	Optional[float]	No	Beta parameter for F-beta metric (default: 1.0)
normalize_arp	bool	No	Whether to normalize Average Recommendation Popularity (default: False)
user_ids	Optional[List[str]]	No	Specific user IDs for RecCasesTable display
display_features	Optional[List[str]]	No	Features to display in RecCasesTable
item_features	Optional[List[str]]	No	Item features for Diversity and Serendipity metrics
user_bias_columns	Optional[List[str]]	No	Columns to analyze for user bias
item_bias_columns	Optional[List[str]]	No	Columns to analyze for item bias
include_tests	bool	No	Whether to include bound tests (default: True)

Outputs

Name	Type	Description
generate_metrics return	Sequence[MetricOrContainer]	A dynamically assembled list of recsys metrics based on configuration and data availability

Usage Examples

from evidently.presets.recsys import RecsysPreset
from evidently.core.report import Report

# Basic recsys evaluation at K=10
report = Report(metrics=[
    RecsysPreset(k=10),
])

# Full evaluation with bias and diversity analysis
report = Report(metrics=[
    RecsysPreset(
        k=5,
        min_rel_score=3,
        beta=1.0,
        item_features=["genre", "category"],
        item_bias_columns=["popularity_bucket"],
        user_bias_columns=["age_group"],
        normalize_arp=True,
    ),
])

report.run(reference_data=ref_df, current_data=curr_df)

Related Pages

• Environment:Evidentlyai_Evidently_Python_Core_Environment