Implementation:Rapidsai Cuml UMAP Metrics

Knowledge Sources	Rapidsai_Cuml
Domains	Machine_Learning, Dimensionality_Reduction
Last Updated	2026-02-08 12:00 GMT

Overview

A comprehensive Python metrics library for evaluating UMAP embedding quality, providing local structure, global structure, fuzzy topological, and persistent homology metrics.

Description

umap_metrics.py provides a collection of functions for quantitatively assessing the quality of dimensionality reduction embeddings. The metrics span four evaluation families:

1. Local Structure Preservation:

• trustworthiness -- Computed via cuML's GPU-accelerated implementation. Measures whether nearby points in the embedding are also nearby in the original space.
• continuity_score -- Measures whether nearby points in the original space remain nearby in the embedding. Handles self-neighbor exclusion and normalizes penalties according to the standard formulation.

2. Global Structure Preservation:

• _compute_geodesic_correlations -- Computes Spearman and Pearson correlations between geodesic distances (shortest paths through the KNN graph) in high-dimensional space and Euclidean distances in the embedding. Uses cuGraph for GPU-accelerated single-source shortest path (SSSP) computations. DEMaP (Distortion of Embedding Metric at All Points) is the Pearson correlation.

3. Fuzzy Simplicial Set Metrics:

• compute_fuzzy_kl_divergence -- KL divergence between aligned Bernoulli edge weights of two fuzzy graphs.
• compute_fuzzy_kl_sym -- Symmetric KL divergence: KL(P||Q) + KL(Q||P).
• compute_fuzzy_js_divergence -- Jensen-Shannon divergence between aligned edge weights.
• compute_edge_jaccard -- Jaccard index over undirected edges with weight above a threshold.
• compute_fuzzy_simplicial_set_metrics -- Aggregates symmetric KL, Jaccard, and row-sum L1 between reference and cuML fuzzy graphs.

4. Cross-Implementation Comparison:

• compute_knn_metrics -- Average neighbor recall and mean absolute distance error between two KNN results.
• compare_spectral_embeddings -- Compares UMAP spectral_layout with cuML SpectralEmbedding using Procrustes RMSE and per-dimension correlations.

5. Topology Preservation:

• Persistent homology via ripser computes Betti numbers (H0 and H1) for both high- and low-dimensional data.

KNN Construction Helpers:

• _build_knn_with_umap -- Builds KNN using scikit-learn NearestNeighbors (brute force) or UMAP's pynndescent backend.
• _build_knn_with_cuvs -- Builds KNN using cuVS brute_force, nn_descent, or all_neighbors (multi-GPU) backends.

Usage

Use these functions to evaluate and compare UMAP embeddings during algorithm development, benchmarking, or quality assurance testing.

Code Reference

Source Location

• Repository: Rapidsai_Cuml
• File: python/cuml/umap_dev_tools/umap_metrics.py

Signature

def compute_knn_metrics(
    knn_graph_a, knn_graph_b, n_neighbors: int
) -> Tuple[float, float]:

def compare_spectral_embeddings(
    fuzzy_graph_cpu, n_components=2, n_neighbors=15, random_state=42
) -> dict:

def continuity_score(
    hr_indices: np.ndarray, lr_indices: np.ndarray,
    n_total: int, k_including_self: int
) -> float:

def compute_fuzzy_kl_divergence(g1, g2, eps=1e-8, average=False) -> float:
def compute_fuzzy_kl_sym(g1, g2, eps=1e-8, average=False) -> float:
def compute_fuzzy_js_divergence(g1, g2, eps=1e-8, average=False) -> float:
def compute_edge_jaccard(g1, g2, eps=0.0) -> float:
def compute_fuzzy_simplicial_set_metrics(ref_fss_graph, cu_fss_graph) -> Tuple:

def compute_simplicial_set_embedding_metrics(
    high_dim_data, embedding, k, metric,
    skip_topology_preservation=False
) -> dict:

def _build_knn_with_umap(X, k, metric, backend) -> Tuple[np.ndarray, np.ndarray]:
def _build_knn_with_cuvs(X, k, metric, backend) -> Tuple[np.ndarray, np.ndarray]:

Import

from umap_metrics import (
    compute_knn_metrics,
    compute_simplicial_set_embedding_metrics,
    compute_fuzzy_simplicial_set_metrics,
    _build_knn_with_umap,
    _build_knn_with_cuvs,
)

I/O Contract

Inputs

Name	Type	Required	Description
high_dim_data	array-like	Yes	Input data in high-dimensional space, shape (n_samples, n_features)
embedding	array-like	Yes	Low-dimensional embedding, shape (n_samples, n_components)
k	int	Yes	Number of nearest neighbors
metric	str	Yes	Distance metric (e.g., "euclidean", "cosine")
skip_topology_preservation	bool	No	If True, skip persistent homology (default: False)
knn_graph_a / knn_graph_b	Tuple[ndarray, ndarray]	Yes (for compare)	KNN results as (distances, indices) arrays

Outputs

Name	Type	Description
metrics	dict	Dictionary of computed metrics including trustworthiness, continuity, geodesic correlations, DEMaP, fuzzy KL divergences, and Betti numbers
avg_recall	float	Average KNN recall between two graph implementations
mae_dist	float	Mean absolute distance error between matching neighbors
kl_sym	float	Symmetric KL divergence between fuzzy graphs
jacc	float	Edge Jaccard index between fuzzy graphs
row_l1	float	Mean relative L1 distance of row sums between fuzzy graphs

Usage Examples

import numpy as np
from umap_metrics import (
    compute_simplicial_set_embedding_metrics,
    compute_knn_metrics,
)

# Evaluate a single embedding
X = np.random.randn(500, 50).astype(np.float32)
embedding = np.random.randn(500, 2).astype(np.float32)  # placeholder

metrics = compute_simplicial_set_embedding_metrics(
    X, embedding, k=15, metric="euclidean",
    skip_topology_preservation=True
)
print(f"Trustworthiness: {metrics['trustworthiness']:.4f}")
print(f"Continuity:      {metrics['continuity']:.4f}")
print(f"DEMaP:           {metrics['demap']:.4f}")

# Compare two KNN implementations
ref_knn = (ref_dists, ref_indices)
cu_knn = (cu_dists, cu_indices)
recall, mae = compute_knn_metrics(ref_knn, cu_knn, n_neighbors=15)
print(f"KNN Recall: {recall:.4f}, Distance MAE: {mae:.6f}")

Related Pages

• Environment:Rapidsai_Cuml_CUDA_GPU