Implementation:Risingwavelabs Risingwave SLT Coverage Checker

Knowledge Sources	RisingWave
Domains	Testing, CI, QualityAssurance
Last Updated	2026-02-09 07:00 GMT

Overview

Python script that analyzes SQLLogicTest (SLT) file coverage by checking which test files are referenced by CI scripts, reporting uncovered files and directory-level coverage statistics.

Description

The e2e_test/check_slt_coverage.py script scans all .slt and .slt.part files under the e2e_test/ directory, then parses CI shell scripts in ci/scripts/ to extract glob patterns that reference test files. It resolves include relationships between test files to propagate coverage transitively (if a covered .slt file includes another file, that included file is also considered covered). The script respects a .coverageignore file for excluding specific patterns from the analysis. Results include per-directory coverage percentages, identification of fully uncovered directories, and detection of files covered by multiple scripts.

Usage

Run as a standalone Python script during CI or local development to ensure all SLT test files are executed in at least one CI pipeline. Supports JSON output for machine consumption.

Code Reference

Source Location

• Repository: risingwave
• File: e2e_test/check_slt_coverage.py
• Lines: L1-649

Signature

# Main entry point
def main():
    parser = argparse.ArgumentParser(
        description="Check SLT test coverage in CI scripts"
    )
    parser.add_argument("--output", "-o", help="Output file for results (JSON format)")
    parser.add_argument("--verbose", "-v", action="store_true", help="Show detailed information")
    parser.add_argument("--dir-analysis", "-d", action="store_true", help="Show directory-level analysis")
    parser.add_argument("--include-ignored", action="store_true", help="Include .coverageignore files")
    # ...

Import

# No installation needed - uses only standard library modules
python3 e2e_test/check_slt_coverage.py [options]

I/O Contract

Inputs

Name	Type	Required	Description
e2e_test/**/*.slt	Files	Yes	All SQLLogicTest files to check for coverage
e2e_test/**/*.slt.part	Files	No	Partial SLT files included by other SLT files
ci/scripts/*.sh	Files	Yes	CI shell scripts to parse for SLT file references
e2e_test/.coverageignore	File	No	Patterns for files to exclude from coverage analysis
--output / -o	CLI flag	No	Path for JSON output file
--verbose / -v	CLI flag	No	Enable detailed output
--dir-analysis / -d	CLI flag	No	Show directory-level coverage breakdown
--include-ignored	CLI flag	No	Include files listed in .coverageignore

Outputs

Name	Type	Description
Console summary	stdout	Coverage statistics: total files, covered count, percentages
JSON results	File (optional)	Structured coverage data with per-file and per-directory details

Key Functions

Function	Purpose
find_all_slt_files()	Discovers all .slt and .slt.part files under e2e_test/, filtering by .coverageignore
extract_slt_patterns_from_script()	Parses a CI shell script to extract glob patterns from risedev slt / sqllogictest / find commands
find_included_files()	Builds include/included-by maps by scanning SLT files for include directives
propagate_coverage_through_includes()	Transitively propagates coverage through include chains to any depth
analyze_uncovered_directories()	Groups uncovered files by directory and computes per-directory coverage percentages
expand_glob_pattern()	Resolves glob patterns to concrete file paths

Coverage Propagation Logic

The script implements transitive coverage propagation:

1. A .slt file is directly covered if matched by a glob pattern in a CI script.
2. If a covered file contains include path/to/other.slt, the included file is covered through includes.
3. .slt.part files are covered if any file that includes them is covered.
4. The propagation runs iteratively until no new files are added (handling multi-level include chains).

Usage Examples

Basic Coverage Check

# Run from the repository root
python3 e2e_test/check_slt_coverage.py

Output:

Total SLT files: 850
Total SLT part files: 120
Total files: 970
--- SUMMARY ---
Covered SLT files: 800 (94.12%)
Covered SLT part files: 115 (95.83%)
Total covered files: 915 (94.33%)
  - Directly covered in scripts: 750
  - Covered through includes: 50
  - Covered part files: 115
Uncovered SLT files: 50 (5.88%)

Verbose Output with Directory Analysis

python3 e2e_test/check_slt_coverage.py -v -d

JSON Output for CI Integration

python3 e2e_test/check_slt_coverage.py -o coverage_report.json
# Produces structured JSON with:
# - total_slt_files, covered_files_count, uncovered_files_count
# - per-script coverage details
# - per-directory coverage percentages
# - list of uncovered files

Related Pages

Implements Principle

• Principle:Risingwavelabs_Risingwave_Test_Coverage_Assurance

Related Implementations

• Implementation:Risingwavelabs_Risingwave_Makefile_Task_Runner -- slt task that runs the SQLLogicTest files this script audits
• Implementation:Risingwavelabs_Risingwave_CI_Docker_Compose -- CI environments where SLT tests are executed
• Implementation:Risingwavelabs_Risingwave_Risedev_Configuration -- CI profiles that select which SLT tests to run