Implementation:Huggingface Diffusers Consolidated Test Report
| Knowledge Sources | |
|---|---|
| Domains | CI, Testing, Reporting |
| Last Updated | 2026-02-13 21:00 GMT |
Overview
Concrete tool for consolidating CI test results from multiple report files into a unified summary with Slack notification support provided by the diffusers CI infrastructure.
Description
The consolidated_test_report.py script aggregates pytest statistics and failure details from multiple test suite report files scattered across subdirectories. It parses `_stats.txt` and `_durations.txt` files to extract passed/failed/skipped counts and slowest test timings. The consolidated data is formatted into a Markdown report and optionally posted to a Slack channel (`diffusers-ci-nightly` by default) with failed test details sent as threaded replies grouped by test suite and ordered by success rate.
Usage
Run this script after CI nightly test runs to consolidate and distribute test results. It is typically triggered by CI automation (GitHub Actions) to post summaries to the `diffusers-ci-nightly` Slack channel. It can also generate local Markdown reports for offline analysis.
Code Reference
Source Location
- Repository: Huggingface_Diffusers
- File: utils/consolidated_test_report.py
- Lines: 1-789
Signature
def parse_stats_file(file_path: str) -> dict:
"""Parse a stats file to extract test statistics.
Returns dict with keys: tests, passed, failed, skipped, slowest_tests.
"""
...
def parse_durations_file(file_path: str) -> list:
"""Parse a durations file to extract test timing information."""
...
def consolidate_reports(reports_dir: str) -> dict:
"""Consolidate all test reports from a directory tree."""
...
def generate_report(consolidated_data: dict) -> str:
"""Generate a Markdown-formatted summary report."""
...
def create_slack_payload(consolidated_data: dict) -> list:
"""Create Slack Block Kit payload for the consolidated report."""
...
def create_failed_tests_by_suite_ordered(consolidated_data: dict) -> list:
"""Group failed tests by suite, ordered by success rate."""
...
def main(args):
"""Main entry point — consolidate, report, and optionally notify Slack."""
...
Import
# CLI script — not imported as a module:
# python utils/consolidated_test_report.py --reports_dir reports --output_file report.md
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| reports_dir | str | Yes | Directory containing test report files (searched recursively) |
| slack_channel_name | str | No | Slack channel for notification (default: `diffusers-ci-nightly`) |
| output_file | str | No | Path to save the consolidated Markdown report |
| SLACK_API_TOKEN | env var | No | Slack API token for sending notifications |
Outputs
| Name | Type | Description |
|---|---|---|
| Markdown report | str/File | Consolidated test summary with pass/fail/skip counts per suite |
| Slack message | API call | Block Kit formatted summary posted to Slack channel |
| Thread replies | API calls | Failed test details grouped by suite, posted as thread replies |
Usage Examples
Generate Local Report
# Consolidate test reports and save to file
python utils/consolidated_test_report.py \
--reports_dir reports/ \
--output_file consolidated_report.md
CI Nightly Usage with Slack
# With SLACK_API_TOKEN set in environment
export SLACK_API_TOKEN="xoxb-..."
python utils/consolidated_test_report.py \
--reports_dir reports/ \
--slack_channel_name diffusers-ci-nightly \
--output_file report.md