Implementation:Evidentlyai Evidently CLI Report

Knowledge Sources	Evidentlyai_Evidently
Domains	CLI, Reports, Data Monitoring, Testing
Last Updated	2026-02-14 12:00 GMT

Overview

CLI command that runs an Evidently report from a YAML/JSON configuration file, applying descriptors and metrics to a dataset with optional test summaries and output saving.

Description

This module provides the report CLI subcommand and its supporting classes/functions:

ReportConfig is a configuration model extending _Config that defines the full specification for a CLI-driven report run. It holds lists of Descriptor, EvidentlyOption, MetricOrContainer, metadata, and tags. Its to_report() method constructs an evidently.Report instance from the configuration.

run_report is the main Typer command. It:

1. Loads the report configuration from a file path
2. Loads the input dataset from a URI (supporting various data sources via _URI)
3. Applies descriptors to the dataset if configured
4. If metrics are defined, optionally loads a reference dataset, constructs and runs the report, and saves the snapshot
5. If only descriptors are defined (no metrics), saves just the dataset
6. Optionally runs a test summary using RowTestSummary, printing color-coded pass/fail results and exiting with code 1 if any tests fail

_run_summary_report and _print_summary_report are helper functions that run a RowTestSummary metric on a dataset/snapshot and print color-coded results grouped by TestStatus.

Usage

Use this command to run Evidently reports from the command line in CI/CD pipelines, batch processing, or automated monitoring workflows. The config file allows declarative specification of descriptors, metrics, metadata, and test conditions.

Code Reference

Source Location

• Repository: Evidentlyai_Evidently
• File: src/evidently/cli/report.py

Signature

class ReportConfig(_Config):
    descriptors: List[Descriptor] = []
    options: List[EvidentlyOption] = []
    metrics: List[MetricOrContainer] = []
    metadata: Dict[str, MetadataValueType] = {}
    tags: List[str] = []
    include_tests: bool = False

    def to_report(self) -> Report:

@app.command("report")
def run_report(
    config_path: str = Argument(..., help="Report configuration path"),
    input_path: str = Argument(..., help="Input dataset URI", metavar="input"),
    output: str = Argument(..., help="Output URI"),
    reference_path: Optional[str] = Option(default=None, help="reference dataset"),
    dataset_name: str = Option("CLI run", help="Name of dataset"),
    test_summary: bool = Option(False, help="Run tests summary"),
    save_dataset: bool = Option(True, help="Save output dataset"),
    save_report: bool = Option(True, help="Save output report"),
):

def _run_summary_report(dataset: Dataset) -> bool:

def _print_summary_report(summary: Snapshot) -> bool:

Import

from evidently.cli.report import run_report, ReportConfig

I/O Contract

Inputs

Name	Type	Required	Description
config_path	str	Yes	Path to YAML/JSON report configuration file
input_path	str	Yes	URI for the input dataset (local path or remote)
output	str	Yes	URI for saving the output (dataset or snapshot)
reference_path	Optional[str]	No	URI for an optional reference dataset for comparison
dataset_name	str	No (default: "CLI run")	Display name for the dataset
test_summary	bool	No (default: False)	Whether to run and print test summary results
save_dataset	bool	No (default: True)	Whether to save the output dataset
save_report	bool	No (default: True)	Whether to save the output report snapshot

Outputs

Name	Type	Description
(side effect)	None	Saves report snapshot and/or dataset to the output URI; prints color-coded test results if test_summary is enabled
(exit code)	int	Exits with code 1 if any tests fail when test_summary is True

Usage Examples

# Run a report with default options
evidently report config.yaml ./data/input.csv ./output/

# Run with reference dataset and test summary
evidently report config.yaml ./data/current.csv ./output/ \
    --reference-path ./data/reference.csv --test-summary

# Run descriptors only, save dataset without report
evidently report descriptors_config.yaml ./data/input.csv ./output/ \
    --no-save-report --dataset-name "Production batch"

Related Pages

• Environment:Evidentlyai_Evidently_Python_Core_Environment
• Evidentlyai_Evidently_Metric_Container - MetricOrContainer type used in report configuration
• Evidentlyai_Evidently_Core_Tests - Test infrastructure used for test summaries