Implementation:Huggingface Transformers Check Bad Commit
| Knowledge Sources | |
|---|---|
| Domains | CI_CD, Debugging |
| Last Updated | 2026-02-13 20:00 GMT |
Overview
Concrete tool for identifying the specific git commit that introduced a test failure using automated git bisect.
Description
The check_bad_commit.py utility automates the process of finding which commit broke a specific test. It generates a temporary Python script that installs the repository and runs a target test with pytest --flake-finder. It uses git bisect run to binary-search through commit history between a known good commit and a bad commit. When the culprit commit is found, it queries the GitHub API to find the associated pull request and its author. The is_bad_commit function can also verify a single commit in isolation.
Usage
Use this tool when a CI test starts failing and you need to pinpoint which commit caused the regression. Provide a known good commit SHA and a bad commit SHA along with the failing test path.
Code Reference
Source Location
- Repository: Huggingface_Transformers
- File: utils/check_bad_commit.py
- Lines: 1-312
Signature
def create_script(test: str) -> str:
"""Generate a temporary test runner script for git bisect."""
def is_bad_commit(commit_id: str, test: str) -> bool:
"""Check if a specific commit causes the given test to fail."""
def find_bad_commit(good: str, bad: str, test: str) -> str:
"""Use git bisect to find the first bad commit between good and bad."""
def get_commit_info(commit_id: str) -> Dict:
"""Query GitHub API for PR info associated with a commit."""
Import
python utils/check_bad_commit.py --good abc123 --bad def456 \
--test tests/models/bert/test_modeling_bert.py::BertModelTest::test_forward
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| --good | str | Yes | Known good commit SHA |
| --bad | str | Yes | Known bad commit SHA |
| --test | str | Yes | Full pytest test path to check |
Outputs
| Name | Type | Description |
|---|---|---|
| Bad commit SHA | stdout | The commit that introduced the failure |
| PR info | stdout | Associated pull request number and author |
Usage Examples
Finding the Bad Commit
# Find which commit broke a test
python utils/check_bad_commit.py \
--good v4.35.0 \
--bad main \
--test tests/models/bert/test_modeling_bert.py::BertModelTest::test_forward
# Check if a specific commit is bad
python utils/check_bad_commit.py \
--commit abc123 \
--test tests/models/bert/test_modeling_bert.py::BertModelTest::test_forward