Workflow:Evidentlyai Evidently Monitoring Dashboard Setup
| Knowledge Sources | |
|---|---|
| Domains | ML_Ops, Monitoring, Dashboard |
| Last Updated | 2026-02-14 10:00 GMT |
Overview
End-to-end process for setting up an Evidently monitoring dashboard by creating a Workspace, configuring a Project with dashboard panels, running periodic Reports, and adding snapshots for time-series visualization in the Evidently UI.
Description
This workflow covers the standard procedure for establishing a continuous monitoring dashboard using Evidently's built-in UI system. It involves creating a local or remote Workspace, defining a Project with configured dashboard panels (counters, line plots, histograms, test suite panels), generating periodic Report snapshots from incoming data batches, and adding them to the project for persistent time-series visualization. The Evidently UI service then renders these snapshots as interactive monitoring dashboards.
Goal: A fully configured Evidently monitoring project with dashboard panels that display metric trends, test results, and data distributions over time, accessible through the Evidently UI web service.
Scope: From workspace creation through project configuration, dashboard panel setup, periodic report generation, to snapshot storage and UI visualization.
Strategy: Uses Evidently's Workspace/Project hierarchy to organize monitoring state, DashboardPanel classes to define visualizations, and periodic Report.run() calls to generate snapshots that populate the dashboard over time.
Usage
Execute this workflow when you need to set up ongoing monitoring for an ML or LLM system and want to use Evidently's built-in dashboard UI (rather than external tools like Grafana). This provides a self-contained monitoring solution with support for metric trends, test suite results, distribution histograms, and counter summaries, all viewable through a web interface.
Execution Steps
Step 1: Create Workspace
Initialize an Evidently Workspace that serves as the storage root for all projects, snapshots, and datasets. A Workspace can be local (file-system based) or remote (connecting to an Evidently Cloud or self-hosted service).
Key considerations:
- Local workspaces store data in a directory on disk
- Remote workspaces connect to an Evidently service via HTTP
- The workspace path determines where all project data is persisted
Step 2: Create and Configure Project
Create a Project within the Workspace. The Project is the organizational unit that contains a dashboard, snapshots, and datasets. Assign a name and description to identify the monitoring target.
Key considerations:
- Each project has a unique UUID identifier
- The project name and description appear in the UI
- Projects can be retrieved by ID for updates
Step 3: Configure Dashboard Panels
Add dashboard panels to the Project that define how metrics are visualized. Available panel types include counters (single aggregated values), line plots (metric trends over time), histograms (value distributions), and test suite panels (pass/fail summaries).
Key considerations:
- DashboardPanelCounter shows aggregated values (sum, last, average)
- DashboardPanelPlot shows time-series line, bar, or scatter charts
- DashboardPanelHistogram shows value distributions from metrics
- DashboardPanelTestSuite shows test pass/fail results over time
- Each panel references metrics by their metric_id and field_path
- PanelValue objects map metric results to panel data points
- Panels can be filtered by metadata, tags, or specific test conditions
Step 4: Define Monitoring Report
Configure a Report with the metrics that match the dashboard panels. The report defines what gets computed for each data batch. Include metrics for drift, model quality, data statistics, and text evaluations as needed.
Key considerations:
- Report metrics must match the metric_id values referenced in dashboard panels
- Enable include_tests=True to populate test suite dashboard panels
- Descriptors for text evaluation must be applied when creating datasets
Step 5: Generate Periodic Snapshots
For each monitoring period (e.g., daily batch), create an Evidently Dataset from the current data, run the Report against it (with optional reference data), and obtain a Snapshot result. Set the timestamp to reflect the monitoring period.
Pseudocode:
for each batch_period:
current_dataset = Dataset.from_pandas(batch_data, data_definition=schema)
snapshot = report.run(current_data=current_dataset, reference_data=reference,
timestamp=batch_timestamp)
Step 6: Add Snapshots to Project
Add each generated Snapshot to the Project. This persists the metric results and makes them available for the dashboard to render trend visualizations.
Key considerations:
- project.add_snapshot(snapshot) persists the results
- Snapshots are ordered by timestamp for time-series rendering
- Historical snapshots can be backfilled for dashboard initialization
Step 7: Launch and View Dashboard
Start the Evidently UI service pointing to the Workspace, then access the web interface to view the configured monitoring dashboard with all panels populated from the stored snapshots.
Key considerations:
- The CLI command "evidently ui" starts the web service
- Access the dashboard at localhost:8000 by default
- Panels update as new snapshots are added to the project