Workflow:Sdv dev SDV Data quality evaluation
| Knowledge Sources | |
|---|---|
| Domains | Synthetic_Data, Data_Quality, Evaluation, Visualization |
| Last Updated | 2026-02-14 19:00 GMT |
Overview
End-to-end process for evaluating synthetic data quality by comparing it to real data using statistical quality reports, diagnostic checks, and visual comparisons provided by the SDV evaluation module.
Description
This workflow covers the assessment of synthetic data fidelity after generation. The SDV evaluation module wraps the SDMetrics library to provide quality reports (measuring statistical similarity), diagnostic reports (checking data validity), and visualization tools (column distribution plots, column pair plots, and cardinality plots). Reports produce scores on a 0-100% scale across multiple metric categories. The evaluation is available for both single-table and multi-table synthetic data.
Usage
Execute this workflow after generating synthetic data to quantify how well it preserves real data properties. Use it to compare different synthesizer configurations, identify weak columns or relationships, validate that anonymization did not distort distributions, and produce evidence of synthetic data quality for stakeholders or compliance reviews.
Execution Steps
Step 1: Prepare real and synthetic data
Ensure both the real data and synthetic data are available as pandas DataFrames (single-table) or dictionaries of DataFrames (multi-table), along with the corresponding Metadata object. Both datasets must have identical column schemas.
Key considerations:
- Column names and types must match between real and synthetic data
- The Metadata object must accurately describe both datasets
- For multi-table evaluation, all tables must be present in both data dictionaries
- Remove any internal or index columns not described in metadata
Step 2: Generate quality report
Call evaluate_quality with the real data, synthetic data, and metadata. This produces a QualityReport object that scores synthetic data on Column Shapes (marginal distribution similarity) and Column Pair Trends (bivariate correlation preservation).
Key considerations:
- Column Shapes measures how well each column's distribution matches the real data
- Column Pair Trends measures correlation and dependency preservation between column pairs
- The overall score is the average of both category scores
- Verbose mode prints progress and summary scores during generation
- For multi-table data, use the multi-table version which aggregates per-table scores
Step 3: Run diagnostic report
Call run_diagnostic to check data validity. The DiagnosticReport verifies that synthetic data falls within expected ranges, contains valid categories, and respects key uniqueness. This catches structural issues that quality scores might miss.
Key considerations:
- Diagnostic checks include data validity (correct value ranges and categories)
- Reports flag columns where synthetic data falls outside real data bounds
- Diagnostic scores indicate whether synthetic data is structurally sound
- Both single-table and multi-table diagnostic reports are available
Step 4: Visualize column distributions
Use get_column_plot to create side-by-side visual comparisons of individual columns between real and synthetic data. This produces histograms for numerical/datetime columns and bar charts for categorical/boolean columns.
Key considerations:
- Plot type is auto-selected based on column sdtype (distplot for numerical/datetime, bar for categorical)
- Custom plot types can be specified to override auto-selection
- Returns a Plotly figure object that can be displayed or saved
- For multi-table data, specify the table name in addition to column name
Step 5: Visualize column pair relationships
Use get_column_pair_plot to compare bivariate relationships between two columns. This shows scatter plots for numerical pairs, heatmaps for categorical pairs, and box plots for mixed type pairs.
Key considerations:
- Useful for verifying that correlations and dependencies are preserved
- Sample size parameter can limit plotted points for large datasets
- Plot type is auto-selected based on column sdtypes
- Supports scatter, heatmap, box, and violin plot types
Step 6: Analyze cardinality (multi_table only)
For multi-table data, use get_cardinality_plot to compare the distribution of child rows per parent row between real and synthetic data. This verifies that the hierarchical sampling preserved realistic relationship sizes.
Key considerations:
- Requires specifying parent table, child table, and foreign key column
- Compares the count distribution of children per parent key
- Available as bar or distplot visualization
- Only applicable to multi-table synthetic data