Principle:Apache Airflow Task Instance State
| Knowledge Sources | |
|---|---|
| Domains | State_Management, Workflow_Orchestration |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A state machine governing task lifecycle transitions from initial scheduling through execution to terminal states.
Description
Task Instance State Management defines the lifecycle of individual task executions within a DagRun. Each TaskInstance progresses through a well-defined state machine: from none (not yet created) through queued, scheduled, running, and finally to a terminal state (success, failed, up_for_retry, skipped, upstream_failed). The state transitions are governed by dependency checks (trigger rules, upstream states, pool availability) and execution outcomes.
Usage
Understanding task instance states is essential for debugging pipeline failures, configuring retry behavior, implementing custom trigger rules, and monitoring task execution health.
Theoretical Basis
State Machine:
# Task Instance state transitions (simplified)
# none → scheduled (dependencies met)
# scheduled → queued (pool slot available)
# queued → running (executor picks up)
# running → success | failed | up_for_retry
# up_for_retry → scheduled (after retry_delay)
# running → deferred (task defers to trigger)
# deferred → scheduled (trigger fires event)
Trigger Rules: Determine when a task is eligible to run based on upstream states:
- all_success: All upstreams succeeded (default)
- all_failed: All upstreams failed
- one_success: At least one upstream succeeded
- none_failed: No upstream failed (allows skipped)