Implementation:Rapidsai Cuml UMAP Debug Runner

Knowledge Sources	Rapidsai_Cuml
Domains	Machine_Learning, Dimensionality_Reduction
Last Updated	2026-02-08 12:00 GMT

Overview

A command-line Python tool for running, comparing, and evaluating UMAP embedding quality between the reference UMAP implementation and the cuML GPU-accelerated implementation.

Description

run_umap_debug.py is the main entry point for the UMAP developer tools suite. It provides a comprehensive pipeline for evaluating UMAP embedding quality across multiple datasets using multiple quality metrics.

The tool supports three operating modes via the --implementation flag:

• reference -- Runs only the reference (CPU) UMAP implementation from the umap-learn package.
• cuml -- Runs only the cuML GPU-accelerated UMAP implementation.
• both (default) -- Runs both implementations and computes cross-implementation comparison metrics.

The pipeline consists of three main stages per dataset:

1. K-Nearest Neighbors -- Computes the KNN graph using brute force search (reference uses scikit-learn's NearestNeighbors; cuML uses cuVS brute_force).
2. Fuzzy Simplicial Set -- Constructs the fuzzy topological representation from the KNN graph using pre-computed distances and indices.
3. Simplicial Set Embedding -- Optimizes the low-dimensional embedding via stochastic gradient descent with attraction/repulsion forces.

Key functions include:

• run_umap_pipeline(X, implementation, **umap_params) -- Executes the full pipeline and returns the KNN graph, fuzzy graph, spectral initialization, and final embedding.
• compare_implementations(X, **umap_params) -- Runs both implementations and computes quality metrics plus cross-implementation comparison metrics (KNN recall, fuzzy graph KL divergence, Jaccard similarity, row-sum L1 distance).
• print_metrics(metrics, name) -- Formatted console output of all computed metrics.
• main() -- CLI argument parsing, dataset iteration, and optional HTML web report generation.

Computed metrics include trustworthiness, continuity, geodesic Spearman/Pearson correlations, DEMaP, fuzzy KL divergences, and persistent homology Betti numbers.

Usage

Use this tool during development of the cuML UMAP implementation to compare its output against the reference implementation, diagnose quality regressions, and generate visual web reports.

Code Reference

Source Location

• Repository: Rapidsai_Cuml
• File: python/cuml/umap_dev_tools/run_umap_debug.py

Signature

def run_umap_pipeline(X, implementation, **umap_params):
    """Returns: (knn_graph, fuzzy_graph, spectral_init, embedding)"""

def run_implementation(X, implementation, **umap_params):
    """Returns: (knn_graph, fuzzy_graph, spectral_init, embedding, metrics)"""

def compare_implementations(X, **umap_params):
    """Returns: dict with ref_*/cuml_* keys for all pipeline outputs and metrics"""

def print_metrics(metrics, name):
    """Print formatted metrics to stdout."""

def main():
    """CLI entry point."""

Import

from run_umap_debug import run_umap_pipeline, compare_implementations
# Or run as CLI:
# python run_umap_debug.py --implementation both --dataset all --web-report

I/O Contract

Inputs

Name	Type	Required	Description
--implementation	string	No	"reference", "cuml", or "both" (default: "both")
--dataset	string	No	Dataset name or "all" (default: "all")
--list-datasets	flag	No	List available datasets and exit
--web-report	flag	No	Generate an interactive HTML report (default: disabled)
--metric	string	No	Distance metric for KNN (default: "euclidean")

Outputs

Name	Type	Description
Console metrics	stdout	Formatted quality metrics for each dataset and implementation
umap_quality_assessment_results.html	HTML file	Interactive web report with Plotly visualizations (when --web-report is set)
Pipeline results	dict	KNN graphs, fuzzy graphs, spectral initializations, embeddings, and metrics

Usage Examples

# Command-line usage:
# Run both implementations on all datasets with web report
python run_umap_debug.py --implementation both --dataset all --web-report

# Run cuML only on a specific dataset
python run_umap_debug.py --implementation cuml --dataset mnist

# List available datasets
python run_umap_debug.py --list-datasets

# Programmatic usage:
import numpy as np
from run_umap_debug import run_umap_pipeline, compare_implementations

X = np.random.randn(1000, 50).astype(np.float32)

# Run a single implementation
knn, fuzzy, spectral, embedding = run_umap_pipeline(
    X, implementation="cuml", n_neighbors=15, metric="euclidean"
)

# Compare both implementations
results = compare_implementations(X, n_neighbors=15, metric="euclidean")
print(f"Trustworthiness (cuml): {results['metrics']['trustworthiness']:.4f}")
print(f"KNN Recall: {results['metrics']['avg_knn_recall']:.4f}")

Related Pages

• Environment:Rapidsai_Cuml_CUDA_GPU