Principle:Huggingface Diffusers Selective Test Execution
| Knowledge Sources | |
|---|---|
| Domains | CI, Testing, Dependency_Analysis |
| Last Updated | 2026-02-13 21:00 GMT |
Overview
Principle for minimizing CI execution time by computing the minimal set of tests impacted by a code change through static import dependency analysis.
Description
Selective test execution solves the scaling problem of CI in large repositories: running the full test suite on every PR is prohibitively expensive when there are hundreds of test files covering different models, pipelines, and features. Instead, the system analyzes which source files changed in a PR, builds a reverse dependency graph of Python imports, and transitively identifies all test files that could be affected by those changes. When too many tests are selected (indicating a core library change), it falls back to running only "important pipeline" tests. Commit message flags provide escape hatches for forcing all tests or skipping CI entirely.
Usage
Apply this principle in CI pipelines to determine which tests to run on a pull request. It is most effective in repositories with a large number of isolated test files where changes typically affect a small subset. The approach requires that test files have clear import dependencies on the modules they test.
Theoretical Basis
Selective test execution builds on static dependency analysis:
- Diff computation: Identify changed files via git diff between PR branch and base, filtering out comment-only changes
- Dependency graph: Build a forward dependency map (module A imports module B) by parsing import statements
- Reverse mapping: Invert the graph to get "module B is depended on by modules A, C, D"
- Transitive closure: Recursively follow reverse dependencies to find all transitively affected modules
- Test filtering: From affected modules, keep only test files; apply importance-based pruning if too many
Pseudo-code Logic:
# Abstract selective test execution flow
changed_files = git_diff(base_branch, pr_branch)
changed_files = filter_comment_only_changes(changed_files)
dependency_graph = {}
for module in all_modules:
dependency_graph[module] = parse_imports(module)
reverse_deps = invert(dependency_graph)
affected_modules = set()
for f in changed_files:
affected_modules |= transitive_closure(reverse_deps, f)
tests_to_run = [m for m in affected_modules if is_test_file(m)]
if len(tests_to_run) > THRESHOLD:
tests_to_run = filter_to_core_pipelines(tests_to_run)