Principle:Mlflow Mlflow Code Quality Linting
| Knowledge Sources | |
|---|---|
| Domains | Static Analysis, Code Quality |
| Last Updated | 2026-02-13 20:00 GMT |
Overview
AST-based custom linting with a symbol index, configurable rule engine, and scope-aware violation detection across source files, notebooks, and documentation code blocks.
Description
A project-specific linter extends standard formatting and style tools by enforcing domain-specific coding conventions that general-purpose linters cannot express. The system consists of three interacting components:
The Linter Engine parses source code into an abstract syntax tree and walks it with a visitor pattern, checking each node against a collection of rules. It supports multiple file types: Python source files are parsed directly; Jupyter notebooks are decomposed into individual code cells that are linted independently; reStructuredText and Markdown documentation files have their embedded code blocks extracted and linted as standalone examples. The engine maintains an import resolver to track how names map to their fully qualified definitions, enabling rules to reason about the actual module a symbol originates from rather than just its local alias. A stack tracks the current nesting context (function, class, module level), allowing rules to behave differently depending on scope. Inline disable comments (clint: disable=rule-name) and per-file ignore configurations suppress specific violations where exceptions are warranted. Unused disable comments are themselves flagged as violations.
The Symbol Index provides a pre-built lookup table mapping fully qualified symbol names to their function signatures (argument names, presence of *args/**kwargs, positional-only and keyword-only parameters). It is constructed by parsing all project Python files in parallel using a process pool, extracting function definitions and import chains via AST visitors. The index is serialized to disk and loaded at lint time, enabling the linter to resolve function calls across module boundaries and verify that keyword arguments in code examples actually exist in the referenced function's signature.
Individual Rules are self-contained classes, each implementing a check for a specific convention. Rules range from simple naming checks (test function name typos, class naming conventions) to sophisticated pattern-matching detections. One notable rule detects opportunities to use the walrus operator: it identifies patterns where a variable is assigned, immediately tested for truthiness in an if statement, used only within that if block, and not referenced in elif, else, or subsequent statements. The rule also verifies the refactored line would not exceed the line length limit.
Usage
This principle applies during pre-commit checks, continuous integration, and interactive development. It catches project-specific anti-patterns (lazy imports of built-in modules, incorrect type annotations, forbidden top-level imports in specific files, RST-style docstrings, markdown links in docstrings) that would otherwise require manual code review to detect.
Theoretical Basis
The linting architecture follows a parse-visit-report pipeline:
For each file:
1. Determine file type (Python, notebook, RST/Markdown)
2. For notebooks: iterate cells, parse each code cell as a module
For documentation: extract code blocks via regex/state-machine parsing
For Python: parse the entire file as a module
3. Walk the AST with the Linter visitor:
- Maintain a scope stack and import resolver
- For each node, invoke applicable rule checks
- For each potential violation:
a. Verify the rule is in the config's select set
b. Check for line-level disable comments
c. Check for per-file ignores
d. If not suppressed, record the violation with file path, position, and rule info
4. Post-visit: check for unused lazy modules, unused disable comments
5. Collect and format all violations
The symbol index construction uses a parallel extract-and-merge strategy:
1. List all project Python files via git ls-files
2. For each file (in parallel via process pool):
a. Parse to AST
b. Walk top-level imports to build an import mapping (alias -> canonical name)
c. Walk function and class definitions to build a signature mapping
3. Merge all per-file mappings into a single global index
4. Serialize to disk for reuse
The walrus operator detection rule applies a precondition chain:
Given a pair of consecutive statements (prev_stmt, if_stmt):
1. prev_stmt must be a single-target, single-line assignment to a Name
2. if_stmt.test must be a Name referencing the same variable
3. The variable must be used (loaded) within if_stmt.body
4. The variable must NOT be used in elif/else branches
5. The variable must NOT be used in statements following the if
6. The refactored line must not exceed the column width limit
If all conditions hold, flag prev_stmt for walrus operator refactoring.