Principle:Apache Dolphinscheduler Task Dispatch Coordination
| Knowledge Sources | |
|---|---|
| Domains | Distributed_Systems, Task_Scheduling |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
A worker-group-aware task dispatch coordination mechanism that routes tasks to the correct worker group dispatcher with support for delayed dispatch and load-balanced worker selection.
Description
The Task Dispatch Coordination principle defines how DolphinScheduler routes tasks from the master to workers. The WorkerGroupDispatcherCoordinator maintains a map of WorkerGroupDispatcher instances, one per active worker group. When a task is submitted for dispatch, the coordinator routes it to the appropriate WorkerGroupDispatcher based on the task's target worker group. Each dispatcher uses a TaskDispatchableEventBus (a delay queue) to manage dispatch timing and a load balancer to select the optimal worker within the group.
This architecture supports multi-tenancy through worker groups, allowing different workloads to be isolated on different worker clusters while sharing a single master for orchestration.
Usage
Task dispatch is triggered automatically by the workflow engine when a task becomes ready for execution (all upstream dependencies satisfied). The task's workerGroup field determines which worker group dispatcher handles it.
Theoretical Basis
The dispatch coordination follows a Router-Dispatcher Pattern:
- Router: WorkerGroupDispatcherCoordinator routes tasks by worker group name
- Dispatcher: WorkerGroupDispatcher manages a queue of tasks for one group
- Queue: TaskDispatchableEventBus provides delayed dispatch capability
- Load Balancer: Selects optimal worker (Random, RoundRobin, FixedWeighted, DynamicWeighted)
dispatchTask(task, delayMills):
groupName = task.getWorkerGroup()
dispatcher = dispatchers.get(groupName)
dispatcher.addToQueue(task, delayMills)
// In WorkerGroupDispatcher thread:
while running:
task = eventBus.take() // blocks until task is ready
worker = loadBalancer.select(group) // choose optimal worker
rpcClient.dispatchTask(worker, task)