Implementation:Evidentlyai Evidently Legacy Recsys Preset

Knowledge Sources	Evidentlyai_Evidently
Domains	Recommender Systems, Machine Learning, Model Monitoring
Last Updated	2026-02-14 12:00 GMT

Overview

The RecsysPreset class is a metric preset that bundles a comprehensive set of recommender system evaluation metrics for use in Evidently reports.

Description

RecsysPreset extends MetricPreset and generates an extensive list of recommendation-related metrics via its generate_metrics method. The preset is parameterized by k (top-k cutoff) and adapts its metric list based on available data.

Core metrics (always included):

PrecisionTopKMetric -- precision at k
RecallTopKMetric -- recall at k
FBetaTopKMetric -- F-beta score at k
MAPKMetric -- mean average precision at k
NDCGKMetric -- normalized discounted cumulative gain at k
MRRKMetric -- mean reciprocal rank at k
HitRateKMetric -- hit rate at k
RecCasesTable -- recommendation cases table for inspection
PersonalizationMetric -- recommendation personalization score

Conditional metrics (based on data availability):

PopularityBias -- added when training data is available (current_train_data in additional_data)
ScoreDistribution -- added when the recommendation type is RecomType.SCORE
DiversityMetric -- added when item_features are provided
SerendipityMetric -- added when both item_features and training data are available
NoveltyMetric -- added when training data is available
ItemBiasMetric -- added for each column in item_bias_columns when training data is available
UserBiasMetric -- added for each column in user_bias_columns when training data is available

All core ranking metrics accept min_rel_score (minimum relevance score threshold) and no_feedback_users (whether to include users without feedback) parameters.

Usage

Use this preset for comprehensive evaluation of recommender system models. It requires specifying the top-k value and adapts automatically based on what data (training data, item features, score type) is available.

Code Reference

Source Location

Repository: Evidentlyai_Evidently
File: src/evidently/legacy/metric_preset/recsys.py

Signature

class RecsysPreset(MetricPreset):
    class Config:
        type_alias = "evidently:metric_preset:RecsysPreset"

    k: int
    min_rel_score: Optional[int]
    no_feedback_users: bool
    normalize_arp: bool
    user_ids: Optional[List[Union[int, str]]]
    display_features: Optional[List[str]]
    item_features: Optional[List[str]]
    user_bias_columns: Optional[List[str]]
    item_bias_columns: Optional[List[str]]

    def __init__(self, k: int,
                 min_rel_score: Optional[int] = None,
                 no_feedback_users: bool = False,
                 normalize_arp: bool = False,
                 user_ids: Optional[List[Union[int, str]]] = None,
                 display_features: Optional[List[str]] = None,
                 item_features: Optional[List[str]] = None,
                 user_bias_columns: Optional[List[str]] = None,
                 item_bias_columns: Optional[List[str]] = None): ...

    def generate_metrics(self, data_definition: DataDefinition,
                         additional_data: Optional[Dict[str, Any]]) -> List[AnyMetric]: ...

Import

from evidently.legacy.metric_preset.recsys import RecsysPreset

I/O Contract

Inputs

Name	Type	Required	Description
k	int	Yes	Top-k cutoff value for ranking metrics
min_rel_score	Optional[int]	No	Minimum relevance score threshold for considering an item as relevant
no_feedback_users	bool	No	Whether to include users without feedback in metric calculations (default: False)
normalize_arp	bool	No	Whether to normalize average recommendation popularity (default: False)
user_ids	Optional[List[Union[int, str]]]	No	Specific user IDs for the RecCasesTable metric
display_features	Optional[List[str]]	No	Feature columns to display in the RecCasesTable
item_features	Optional[List[str]]	No	Item feature columns for diversity and serendipity metrics
user_bias_columns	Optional[List[str]]	No	Columns for user bias analysis (requires training data)
item_bias_columns	Optional[List[str]]	No	Columns for item bias analysis (requires training data)

Outputs

Name	Type	Description
metrics	List[AnyMetric]	A list of 9+ recommendation metric instances, varying based on available data and configuration

Usage Examples

from evidently.legacy.metric_preset.recsys import RecsysPreset

# Basic usage with top-10 recommendations
preset = RecsysPreset(k=10)

# Full configuration with bias analysis and item features
preset = RecsysPreset(
    k=10,
    min_rel_score=3,
    no_feedback_users=False,
    normalize_arp=True,
    user_ids=[1, 2, 3],
    display_features=["title", "genre"],
    item_features=["genre", "category"],
    user_bias_columns=["age_group", "gender"],
    item_bias_columns=["popularity_tier"]
)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment