Principle:Apache Airflow Trigger Handling
| Knowledge Sources | |
|---|---|
| Domains | Async_Execution, Event_Driven |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
An asynchronous event-driven mechanism that allows tasks to defer execution and wait for external events without occupying worker slots.
Description
Trigger Handling enables deferrable operators in Airflow. When a task needs to wait for an external condition (API response, file arrival, time-based event), it can defer itself by registering a Trigger. The TriggererJobRunner runs an asyncio event loop that monitors multiple triggers concurrently. When a trigger fires, the associated task resumes execution from where it left off. This frees worker slots during wait periods, significantly improving resource efficiency for I/O-bound workloads.
Usage
Use deferrable operators when tasks spend significant time waiting for external conditions. This is common for sensors, long-running API calls, and event-driven workflows. The triggerer component must be running for deferred tasks to function.
Theoretical Basis
Deferral Pattern:
- Task calls self.defer(trigger=MyTrigger(), method_name="resume_method")
- TaskInstance state changes to deferred and trigger_id is set
- TriggererJobRunner picks up the trigger and runs it in asyncio
- When trigger fires an event, TaskInstance moves back to scheduled
- Task resumes at the specified method with the event data
Concurrency Model:
- Triggers run as asyncio coroutines in a shared event loop
- Many triggers execute concurrently on a single triggerer process
- Trigger kwargs are encrypted (Fernet) in the database for security