Implementation:Apache Airflow Provider Metadata Registry

Knowledge Sources	Apache_Airflow
Domains	Provider_Management, Registry
Last Updated	2026-02-08 21:00 GMT

Overview

Auto-generated JSON registry mapping every Apache Airflow provider package to its complete version history, associated Airflow versions, and release dates, serving as the canonical source of provider compatibility data.

Description

The generated/provider_metadata.json file is a 14,608-line auto-generated JSON file that serves as the definitive registry for all Airflow provider packages. For each provider, it records every released version along with:

• Associated Airflow version: The minimum compatible Airflow version at the time of that provider release.
• Release date: ISO 8601 timestamp of when the provider version was published.

The file is organized as a top-level JSON object where each key is a provider name (e.g., "airbyte", "amazon", "google") and the value is another object mapping version strings to their metadata. This structure enables:

• Determining which provider versions are compatible with a given Airflow installation.
• Tracking the release timeline of any provider.
• Understanding the co-evolution of providers and core Airflow versions.
• Automated tooling for dependency resolution and compatibility checks.

The file covers 70+ providers, with some providers (like amazon, google) having dozens of version entries spanning from Airflow 2.0.x through the current 3.x series.

Usage

This file is consumed by build tooling, CI pipelines, and documentation generators. It is auto-generated and should not be manually edited. Tooling can parse it to determine compatible provider versions for any given Airflow release.

Code Reference

Source Location

• Repository: Apache_Airflow
• File: generated/provider_metadata.json
• Lines: 14,608

Signature

{
    "airbyte": {
        "1.0.0": {
            "associated_airflow_version": "2.0.2",
            "date_released": "2021-04-06T23:45:20Z"
        },
        "2.0.0": {
            "associated_airflow_version": "2.1.1",
            "date_released": "2021-06-23T12:50:28Z"
        },
        "3.2.0": {
            "associated_airflow_version": "2.5.0",
            "date_released": "2022-11-18T10:44:03Z"
        }
    },
    "amazon": {
        "1.0.0": {
            "associated_airflow_version": "2.0.2",
            "date_released": "2021-04-06T23:45:20Z"
        }
    },
    "google": {
        "1.0.0": {
            "associated_airflow_version": "2.0.2",
            "date_released": "2021-04-06T23:45:20Z"
        }
    }
}

I/O Contract

Inputs

Name	Type	Required	Description
Provider releases	Release metadata	Yes	Version information from each provider package release process
Airflow version mapping	Version constraints	Yes	The minimum Airflow version each provider release was tested against
Release timestamps	ISO 8601 datetime	Yes	Publication date of each provider version on PyPI

Outputs

Name	Type	Description
Version compatibility matrix	JSON object	Nested JSON mapping provider names to version objects containing associated_airflow_version and date_released

Usage Examples

Querying Provider Compatibility

import json

# Load the provider metadata registry
with open("generated/provider_metadata.json") as f:
    registry = json.load(f)

# Find all versions of the amazon provider
amazon_versions = registry["amazon"]
for version, meta in amazon_versions.items():
    print(f"amazon {version}: airflow>={meta['associated_airflow_version']} "
          f"(released {meta['date_released']})")

# Find providers compatible with Airflow 2.5.0
from packaging.version import Version

target = Version("2.5.0")
for provider, versions in registry.items():
    compatible = [
        v for v, m in versions.items()
        if Version(m["associated_airflow_version"]) <= target
    ]
    if compatible:
        latest = max(compatible, key=Version)
        print(f"{provider}: latest compatible version = {latest}")

JSON Structure Schema

Field	Type	Description
{provider_name}	object	Top-level key for each provider (e.g., "airbyte", "amazon")
{provider_name}.{version}	object	Version entry containing release metadata
associated_airflow_version	string	Minimum compatible Airflow version (e.g., "2.5.0")
date_released	string (ISO 8601)	Release timestamp (e.g., "2022-11-18T10:44:03Z")

Related Pages

• Principle:Apache_Airflow_Provider_Package_Scaffolding