Implementation:Apache Airflow Module Loading

Knowledge Sources	Apache_Airflow
Domains	Core_Infrastructure, Module_System
Last Updated	2026-02-08 21:00 GMT

Overview

Provides utilities for dynamic module importing, qualified name resolution, dotpath validation, and entry point discovery with distribution metadata in Apache Airflow.

Description

The airflow_shared.module_loading package is a foundational infrastructure module that supplies five key functions used throughout the Airflow codebase:

• import_string(dotted_path) -- Imports a dotted module path and returns the attribute or class designated by the last name in the path. Splits on the last dot to separate the module path from the attribute name. Raises ImportError if the module cannot be found or does not define the requested attribute. Currently only supports top-level classes (not nested classes).

• qualname(o, use_qualname=False, exclude_module=False) -- Converts an object, callable, or class to its fully qualified string name. By default returns a string importable by import_string (i.e., includes the module path). With exclude_module=True, returns only the qualified name without the module prefix, useful for stable identification across deployments where module paths may vary. Handles functools.partial objects by recursing into the wrapped function.

• iter_namespace(ns) -- Wraps pkgutil.iter_modules to iterate over the sub-modules of a given namespace package.

• is_valid_dotpath(path) -- Validates whether a string follows the valid dotted path format (e.g., package.subpackage.module) using a regular expression. Each segment must start with a letter or underscore and can contain letters, digits, or underscores.

• entry_points_with_dist(group) -- Retrieves entry points of a specified group along with their distribution metadata. Unlike the standard importlib.metadata.entry_points(), this returns (EntryPoint, Distribution) tuples. The results are cached using functools.cache for performance. Note that multiple distributions may map to the same package if loaded from different sys.path entries.

The module also re-exports find_path_from_directory from the file_discovery submodule.

Usage

These utilities are used throughout Airflow for dynamically loading operators, hooks, sensors, executors, log handlers, and plugins. The import_string function is particularly central, being used anywhere a class path is stored in configuration and needs to be resolved at runtime.

Code Reference

Source Location

• Repository: Apache_Airflow
• File: shared/module_loading/src/airflow_shared/module_loading/__init__.py (158 lines)

Signature

def import_string(dotted_path: str):
    """Import a dotted module path and return the attribute/class designated by the last name."""
    ...

def qualname(o: object | Callable, use_qualname: bool = False, exclude_module: bool = False) -> str:
    """Convert an attribute/class/callable to a string."""
    ...

def iter_namespace(ns: ModuleType):
    """Iterate over sub-modules of a namespace package."""
    ...

def is_valid_dotpath(path: str) -> bool:
    """Check if a string follows valid dotpath format."""
    ...

EPnD = tuple[metadata.EntryPoint, metadata.Distribution]

def entry_points_with_dist(group: str) -> Iterator[EPnD]:
    """Retrieve entry points of the given group with their distributions."""
    ...

Import

from airflow_shared.module_loading import import_string, qualname, entry_points_with_dist
from airflow_shared.module_loading import iter_namespace, is_valid_dotpath

I/O Contract

Function	Input	Output	Side Effects / Errors
import_string	dotted_path: str (e.g., "my.module.MyClass")	The imported attribute/class	Raises ImportError if module or attribute not found
qualname	Callable, optional flags	str (fully qualified name)	None
iter_namespace	ns: ModuleType	Iterator of (importer, name, ispkg) tuples	May trigger module discovery I/O
is_valid_dotpath	path: str	bool	None
entry_points_with_dist	group: str	Iterator[tuple[EntryPoint, Distribution]]	Cached; first call scans all installed distributions

Usage Examples

Dynamically Loading a Class

from airflow_shared.module_loading import import_string

# Load an executor class from a dotted path
ExecutorClass = import_string("airflow.executors.local_executor.LocalExecutor")
executor = ExecutorClass()

Getting the Qualified Name of an Object

from airflow_shared.module_loading import qualname

name = qualname(my_operator)
# Returns: "airflow.operators.python.PythonOperator"

short_name = qualname(my_operator, exclude_module=True)
# Returns: "PythonOperator"

Validating a Dotted Path

from airflow_shared.module_loading import is_valid_dotpath

is_valid_dotpath("airflow.operators.python")   # True
is_valid_dotpath("not a valid path!")           # False
is_valid_dotpath("123.invalid")                 # False

Discovering Entry Points with Distribution Info

from airflow_shared.module_loading import entry_points_with_dist

for entry_point, dist in entry_points_with_dist("airflow.providers"):
    print(f"{entry_point.name} from {dist.metadata['Name']} {dist.version}")

Related Pages

• Environment:Apache_Airflow_Python_Runtime_Environment