Implementation:Apache Airflow Root Pyproject

Knowledge Sources	Apache_Airflow
Domains	Build_System, Packaging
Last Updated	2026-02-08 21:00 GMT

Overview

Root meta-package configuration for apache-airflow that serves as the umbrella package aggregating apache-airflow-core, apache-airflow-task-sdk, and 100+ optional provider extras into a single installable distribution.

Description

The pyproject.toml at the repository root is a 1,592-line TOML configuration defining the apache-airflow meta-package. Unlike apache-airflow-core which contains the actual implementation, this package acts as an aggregation layer:

• Core dependencies: Only two direct runtime dependencies:
  • apache-airflow-task-sdk<1.3.0,>=1.2.0
  • apache-airflow-core==3.2.0
• No packages: The meta-package itself ships no Python packages (packages = []); it delegates all functionality to its dependencies.
• 100+ provider extras: Maps every Airflow provider as an optional extra:
  • Cloud providers: amazon, google, microsoft.azure
  • Databases: postgres, mysql, snowflake, databricks
  • Messaging: apache.kafka, slack, telegram
  • Orchestration: celery, cncf.kubernetes, docker
  • AI/ML: openai, cohere, pinecone, weaviate
  • Core extras forwarded from airflow-core: all-core, async, graphviz, gunicorn, kerberos, memray, otel, statsd
• Build system: Same Hatchling setup as airflow-core with custom build hook (hatch_build.py).
• Shared distributions: Force-includes 11 shared library packages (configuration, logging, secrets_backend, timezones, etc.) into the sdist via [tool.hatch.build.targets.sdist.force-include].
• Development groups: Defines [dependency-groups] for dev (testing) and docs (documentation building).
• UV configuration: Uses uv package manager with workspace sources for apache-airflow-core and apache-airflow-devel-common.
• Shared distributions list: [tool.airflow] section enumerates all 11 shared distribution packages.

Usage

# Install apache-airflow with specific provider extras
pip install apache-airflow[amazon,google]

# Install with multiple provider groups
pip install apache-airflow[amazon,google,postgres,celery,cncf.kubernetes]

# Install the base package (includes core + task-sdk only)
pip install apache-airflow

# Install all core extras plus specific providers
pip install "apache-airflow[all-core,amazon,google]"

Code Reference

Source Location

• Repository: Apache_Airflow
• File: pyproject.toml
• Lines: 1,592

Signature

[build-system]
requires = [
    "GitPython==3.1.45",
    "hatchling==1.27.0",
    "packaging==25.0",
    # ...
]
build-backend = "hatchling.build"

[project]
name = "apache-airflow"
description = "Programmatically author, schedule and monitor data pipelines"
license = "Apache-2.0"
requires-python = ">=3.10,!=3.14"
version = "3.2.0"

dependencies = [
    "apache-airflow-task-sdk<1.3.0,>=1.2.0",
    "apache-airflow-core==3.2.0",
]

packages = []

[project.optional-dependencies]
"all-core" = ["apache-airflow-core[all]"]
"async" = ["apache-airflow-core[async]"]
"amazon" = ["apache-airflow-providers-amazon>=9.0.0"]
"google" = ["apache-airflow-providers-google>=12.0.0"]
"postgres" = ["apache-airflow-providers-postgres>=5.13.1"]
"celery" = ["apache-airflow-providers-celery>=3.8.3"]
"cncf.kubernetes" = ["apache-airflow-providers-cncf-kubernetes>=9.0.0"]
# ... 100+ additional provider extras

[tool.hatch.build.targets.sdist.force-include]
"../shared/configuration/src/airflow_shared/configuration" = "src/airflow/_shared/configuration"
"../shared/logging/src/airflow_shared/logging" = "src/airflow/_shared/logging"
# ... 11 shared distributions

[dependency-groups]
dev = [
    "apache-airflow-core[all]",
    "apache-airflow-ctl",
    "apache-airflow-devel-common",
    "apache-airflow-task-sdk",
    "apache-airflow[pandas,polars]",
    # ... provider dev dependencies
]
docs = ["apache-airflow-devel-common[docs]"]

[tool.uv]
required-version = ">=0.6.3"

[tool.airflow]
shared_distributions = [
    "apache-airflow-shared-configuration",
    "apache-airflow-shared-logging",
    "apache-airflow-shared-secrets-backend",
    # ... 11 total
]

I/O Contract

Inputs

Name	Type	Required	Description
apache-airflow-core	Package dependency	Yes	Core Airflow engine package (pinned to same version)
apache-airflow-task-sdk	Package dependency	Yes	Task SDK for DAG authoring
Provider packages	Optional dependencies	No	100+ provider packages selectable via extras
Shared distributions	Force-included sources	Yes	11 shared library packages included in sdist

Outputs

Name	Type	Description
Unified apache-airflow package	sdist/wheel	Single installable package that pulls in core + selected providers
Provider extras resolution	Dependency resolution	Installing with [amazon] pulls apache-airflow-providers-amazon>=9.0.0

Usage Examples

Installing With Provider Extras

# Production deployment with AWS and Kubernetes support
pip install "apache-airflow[amazon,cncf.kubernetes,celery,postgres]"

# Data engineering stack
pip install "apache-airflow[google,snowflake,databricks,apache.spark]"

# Minimal install with all core extras
pip install "apache-airflow[all-core]"

Provider Extra Mapping Examples

Extra Name	Provider Package	Minimum Version
amazon	apache-airflow-providers-amazon	>=9.0.0
google	apache-airflow-providers-google	>=12.0.0
celery	apache-airflow-providers-celery	>=3.8.3
cncf.kubernetes	apache-airflow-providers-cncf-kubernetes	>=9.0.0
postgres	apache-airflow-providers-postgres	>=5.13.1
apache.spark	apache-airflow-providers-apache-spark	>=4.11.1

Development Setup

# Install development dependencies
uv sync --group dev

# Build documentation
uv run --group docs build-docs

Related Pages

• Environment:Apache_Airflow_Python_Runtime_Environment