Principle:Apache Airflow Callback Event Notification
| Knowledge Sources | |
|---|---|
| Domains | Event_Driven, Plugin_System |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A plugin-based event notification system that dispatches lifecycle callbacks to registered listeners on task and DAG state changes.
Description
Callback and Event Notification provides a hook-based mechanism for extending Airflow's behavior in response to lifecycle events. The ListenerManager uses the pluggy plugin framework to manage hookspecs (event definitions) and hookimpls (event handlers). Listeners can react to task instance state changes (running, success, failed), DAG run state changes, and other lifecycle events. This enables custom monitoring, alerting, and integration without modifying core Airflow code.
Usage
Use the listener system when you need to react to Airflow events (task completion, failure, etc.) for custom monitoring, alerting, or integration purposes. Register listeners via Airflow plugins or airflow_local_settings.
Theoretical Basis
Observer Pattern (via pluggy):
- HookSpec: Defines the event signature (e.g., on_task_instance_success)
- HookImpl: Concrete listener implementation matching the spec
- PluginManager: Dispatches events to all registered implementations
Event Flow:
# Pseudo-code for listener dispatch
def on_task_state_change(task_instance, new_state):
if new_state == RUNNING:
listener_manager.hook.on_task_instance_running(
previous_state=old_state,
task_instance=task_instance,
)
elif new_state == SUCCESS:
listener_manager.hook.on_task_instance_success(...)
elif new_state == FAILED:
listener_manager.hook.on_task_instance_failed(...)