Jump to content

Connect SuperML | Leeroopedia MCP: Equip your AI agents with best practices, code verification, and debugging knowledge. Powered by Leeroo — building Organizational Superintelligence. Contact us at founders@leeroo.com.

Implementation:Apache Airflow Core Pre Commit Config

From Leeroopedia
Revision as of 14:10, 16 February 2026 by Admin (talk | contribs) (Auto-imported from implementations/Apache_Airflow_Core_Pre_Commit_Config.md)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


Knowledge Sources
Domains CI_CD, Code_Quality
Last Updated 2026-02-08 21:00 GMT

Overview

Airflow-core specific pre-commit hooks configuration that enforces import restrictions, operator argument validation, provider isolation, UI compilation, mypy type checking, and schema validation for the airflow-core package.

Description

The airflow-core/.pre-commit-config.yaml file is a 413-line YAML configuration defining local pre-commit hooks specific to the airflow-core package. Unlike the root config which covers the entire monorepo, these hooks focus on airflow-core code quality and architectural constraints. The hooks are organized into two groups:

Standard pre-commit hooks:

  • check-taskinstance-tis-attrs -- Verifies that TaskInstance and TaskInstanceState have the same attributes
  • check-airflow-k8s-not-used -- Enforces that airflow.kubernetes imports are not used; must use airflow.providers.cncf.kubernetes instead
  • check-common-compat-used-for-openlineage -- Ensures OpenLineage classes are imported from airflow.providers.common.compat
  • check-cncf-k8s-only-for-executors -- Restricts airflow.providers.cncf.kubernetes imports to executor files only
  • check-extra-packages-references -- Validates extras defined in hatch_build.py are listed in documentation
  • check-no-providers-in-core-examples -- Prevents provider imports in core example DAGs (only standard provider allowed)
  • check-only-new-session-with-provide-session -- Enforces NEW_SESSION is only used with @provide_session
  • check-base-operator-partial-arguments -- Validates BaseOperator and partial() arguments match
  • check-decorated-operator-implements-custom-name -- Verifies @task decorators implement custom_operator_name
  • compile-ui-assets / compile-ui-assets-dev -- Build UI assets (manual stage)
  • check-tests-in-the-right-folders -- Validates test file placement
  • lint-json-schema -- Validates config.yml against its JSON schema
  • check-code-deprecations -- Validates deprecation categories in decorators
  • create-missing-init-py-files-tests -- Creates missing __init__.py in test directories
  • check-tests-unittest-testcase -- Prevents inheriting from unittest.TestCase
  • check-no-fab-migrations -- Prevents FAB-related database migrations in core
  • ts-compile-lint-ui -- TypeScript compilation, ESLint, and Prettier for the UI
  • ts-compile-lint-simple-auth-manager-ui -- Same for the simple auth manager UI

Hooks requiring CI image (pre-push/manual stage):

  • mypy-airflow-core -- Run mypy type checking (pre-push stage)
  • generate-openapi-spec -- Generate FastAPI OpenAPI specification
  • check-i18n-json -- Validate i18n translation files
  • update-migration-references -- Update migration reference documentation
  • update-er-diagram -- Update entity-relationship diagram
  • check-default-configuration -- Validate default configuration values
  • check-airflow-version-checks-in-core -- Prevent AIRFLOW_V_* imports in core
  • check-sdk-imports -- Check for SDK imports in core files (with approved exception list)
  • check-schema-defaults -- Validate schema defaults match server-side defaults

Usage

These hooks run automatically when committing files within the airflow-core/ directory. They can also be run manually:

# Run core-specific hooks from the airflow-core directory
cd airflow-core
pre-commit run --all-files

# Run a specific core hook
pre-commit run check-airflow-k8s-not-used --all-files

Code Reference

Source Location

  • Repository: Apache_Airflow
  • File: airflow-core/.pre-commit-config.yaml
  • Lines: 413

Signature

default_stages: [pre-commit, pre-push]
minimum_prek_version: '0.2.0'
default_language_version:
  python: python3
  node: 22.19.0
  golang: 1.24.0
repos:
  - repo: local
    hooks:
      - id: check-taskinstance-tis-attrs
        name: Check that TI and TIS have the same attributes
        entry: ../scripts/ci/prek/check_ti_vs_tis_attributes.py
        language: python
        files: ^src/airflow/models/taskinstance\.py$|^src/airflow/models/taskinstancehistory\.py$

      - id: check-airflow-k8s-not-used
        name: Check airflow.kubernetes imports are not used
        entry: ../scripts/ci/prek/check_airflow_imports.py
          --pattern '^airflow\.kubernetes'
          --message "You should only import kubernetes code from `airflow.providers.cncf.kubernetes`."

      - id: check-no-providers-in-core-examples
        language: pygrep
        name: No providers imports in core example DAGs
        entry: "^\\s*from airflow\\.providers.(?!standard.)"
        files: ^src/airflow/example_dags/.*\.py$

      - id: mypy-airflow-core
        stages: ['pre-push']
        name: Run mypy for airflow-core
        entry: ../scripts/ci/prek/mypy.py

      - id: check-sdk-imports
        name: Check for SDK imports in core files
        entry: ../scripts/ci/prek/check_sdk_imports.py

I/O Contract

Inputs

Name Type Required Description
airflow-core Python files .py files Yes Source files under airflow-core/src/airflow/
airflow-core test files .py files No Test files under airflow-core/tests/
UI source files .ts, .tsx, .js No TypeScript/JavaScript UI files
Configuration files .yml, .json No Config template and schema files

Outputs

Name Type Description
Validation results Exit code + messages Pass/fail status with descriptive error messages
Generated files OpenAPI specs, ER diagrams, migration docs Auto-generated artifacts from manual-stage hooks
UI assets Compiled JS/CSS Built UI bundles (manual stage)

Usage Examples

Import Restriction Check

# This hook prevents direct kubernetes imports in core code
# and enforces using the cncf.kubernetes provider instead
- id: check-airflow-k8s-not-used
  name: Check airflow.kubernetes imports are not used
  language: python
  files: ^src/airflow/.*\.py$
  exclude: ^src/airflow/kubernetes/
  entry: ../scripts/ci/prek/check_airflow_imports.py
    --pattern '^airflow\.kubernetes'
    --message "You should only import kubernetes code from `airflow.providers.cncf.kubernetes`."

Provider Isolation in Example DAGs

# Ensures core example DAGs only import from the standard provider
- id: check-no-providers-in-core-examples
  language: pygrep
  name: No providers imports in core example DAGs
  entry: "^\\s*from airflow\\.providers.(?!standard.)"
  files: ^src/airflow/example_dags/.*\.py$

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment