Implementation:Huggingface Transformers Check Copies
| Knowledge Sources | |
|---|---|
| Domains | Code_Quality, Repository_Maintenance |
| Last Updated | 2026-02-13 20:00 GMT |
Overview
Concrete tool for validating that code marked with # Copied from annotations stays synchronized with its source, and that model lists in README files are consistent.
Description
The check_copies.py utility (1046 lines) parses Python source files for # Copied from annotations, locates the original code block, applies any specified transformations (class/function renaming), and compares against the current copy. For README checks, it extracts the model list from the main README and verifies localized READMEs match. Uses ruff for code formatting normalization before comparison. Supports both check-only and auto-fix modes (--fix_and_overwrite). This is fundamental to how Transformers manages model code that shares structure across implementations.
Usage
Run as part of make check-repo or make fix-repo to validate or auto-fix code copy synchronization.
Code Reference
Source Location
- Repository: Huggingface_Transformers
- File: utils/check_copies.py
- Lines: 1-1046
Signature
def find_code_in_transformers(
object_name: str,
base_path: str = None,
) -> str:
"""Find and return the source code of an object in the transformers package."""
def replace_code(
code: str,
replacements: List[Tuple[str, str]],
) -> str:
"""Apply name replacement transformations to copied code."""
def check_copies(overwrite: bool = False) -> List[str]:
"""Check all '# Copied from' annotations in the codebase."""
def check_full_copies(overwrite: bool = False) -> List[str]:
"""Check full-file copies (entire files marked as copied)."""
Import
python utils/check_copies.py
python utils/check_copies.py --fix_and_overwrite
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| src/transformers/ | Directory | Yes | Source files containing # Copied from annotations |
| --fix_and_overwrite | flag | No | Auto-fix out-of-sync copies |
Outputs
| Name | Type | Description |
|---|---|---|
| Error report | stdout | List of out-of-sync copies |
| Updated files | Python files | Modified source files (if --fix_and_overwrite) |
Usage Examples
Checking Code Copies
# Check for out-of-sync copies
python utils/check_copies.py
# Auto-fix copies
python utils/check_copies.py --fix_and_overwrite
# Part of the standard workflow
make check-repo # checks
make fix-repo # auto-fixes