Principle:Huggingface Transformers Selective Test Execution
| Knowledge Sources | |
|---|---|
| Domains | CI_CD, Testing_Infrastructure |
| Last Updated | 2026-02-13 20:00 GMT |
Overview
Principle of running only the tests affected by code changes by building and traversing an import dependency graph of the codebase.
Description
Selective Test Execution (also known as Test Impact Analysis) determines the minimal set of tests that must run to validate a set of code changes. Rather than running the entire test suite on every change, the system builds a dependency graph by parsing import statements across all source and test files. When a file is modified, all transitively dependent test files are identified and included in the test run. This dramatically reduces CI time in large repositories while maintaining confidence that all affected code paths are tested. The system must handle edge cases like core file modifications (which trigger full CI) and docstring-only changes (which can be skipped).
Usage
Apply this principle in repositories with large test suites (100+ test files) where full test runs are prohibitively slow. Requires a build step that computes the dependency graph before test execution, and a CI system that can accept a dynamic list of test files to run.
Theoretical Basis
The algorithm constructs and traverses a reverse dependency graph:
Step 1: Build forward dependency map
- For each Python file, parse its imports
- Record which modules each file depends on
Step 2: Invert to reverse dependency map
- For each module, record which files depend on it
Step 3: Compute transitive closure
- For each modified file, follow reverse dependencies transitively
- Collect all reached test files
Step 4: Apply filters and thresholds
- Skip docstring-only changes
- Trigger full CI if core files or too many models are affected
- Support override flags (e.g., [test all], [ci skip])
Pseudo-code:
# Abstract algorithm (NOT real implementation)
forward_deps = build_import_graph(all_source_files)
reverse_deps = invert(forward_deps)
modified = get_modified_files(base_branch)
tests_to_run = set()
for file in modified:
tests_to_run |= transitive_closure(reverse_deps, file)
if is_core_change(modified) or len(tests_to_run) > threshold:
tests_to_run = all_tests
write_test_lists(tests_to_run)