Principle:Apache Airflow Provider Package Scaffolding
| Knowledge Sources | |
|---|---|
| Domains | Plugin_System, Package_Management |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A standardized package structure and metadata schema for creating Airflow provider distributions that integrate external services.
Description
Provider Package Scaffolding defines the directory structure, metadata format (provider.yaml), and build configuration (pyproject.toml) required for creating Airflow provider packages. Each provider encapsulates hooks, operators, sensors, triggers, and connection types for integrating with a specific external service. The provider.yaml file declares all components and their metadata, which the ProvidersManager uses for discovery and registration.
Usage
Use this principle when creating a new provider package to integrate Airflow with an external service (database, cloud platform, messaging system, etc.). Follow the standardized structure for compatibility with the Airflow provider ecosystem.
Theoretical Basis
Provider Architecture:
- provider.yaml: Declarative metadata — package name, version, integrations, component lists
- Directory Structure: Standardized layout: hooks/, operators/, sensors/, triggers/, transfers/
- Entry Points: Packages register via apache_airflow_provider entry point
- Discovery: ProvidersManager scans entry points to find all installed providers
Package Lifecycle States:
- incubation: New provider under development
- ready: Production-ready provider
- attic: Deprecated provider (maintenance only)