Heuristic:Apache Dolphinscheduler Load Balancer Strategy Selection
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Distributed_Systems |
| Last Updated | 2026-02-10 10:00 GMT |
Overview
Worker load balancing strategy selection: ROUND_ROBIN (default), RANDOM, FIXED_WEIGHTED_ROUND_ROBIN, or DYNAMIC_WEIGHTED_ROUND_ROBIN with tunable CPU/Memory/ThreadPool weights summing to 100.
Description
DolphinScheduler's Master dispatches tasks to Workers using a pluggable load balancer. Four strategies are available: ROUND_ROBIN (default, even distribution), RANDOM (quick selection via SecureRandom), FIXED_WEIGHTED_ROUND_ROBIN (admin-configured static weights), and DYNAMIC_WEIGHTED_ROUND_ROBIN (real-time weights based on worker resource usage). The dynamic strategy calculates weight as `100 - (cpuWeight * cpuUsage + memWeight * memUsage + threadWeight * threadUsage) / 3` with default weights of CPU=30%, Memory=30%, ThreadPool=40%.
Usage
Apply this heuristic when tuning task distribution across Workers. Use ROUND_ROBIN for homogeneous clusters. Use DYNAMIC_WEIGHTED for heterogeneous clusters or when Workers have uneven resource profiles. The dynamic strategy prevents overloading busy Workers by reducing their selection probability.
The Insight (Rule of Thumb)
- Action: Set `master.worker-load-balancer.type` in `application.yaml`. For dynamic weighting, configure `cpuUsageWeight`, `memoryUsageWeight`, `taskThreadPoolUsageWeight`.
- Value:
- Default strategy: ROUND_ROBIN
- Dynamic weight defaults: CPU=30, Memory=30, ThreadPool=40
- Constraint: Weights must sum to 100 (validated at startup)
- Trade-off: ROUND_ROBIN is simple but ignores resource utilization. DYNAMIC_WEIGHTED adapts to load but requires accurate resource metrics from Workers.
Reasoning
The default ROUND_ROBIN strategy distributes tasks evenly, which is optimal when all Workers have identical resources. In heterogeneous environments, some Workers may be overloaded while others are idle. The dynamic strategy addresses this by weighting ThreadPool usage most heavily (40%) because task queue saturation is the most immediate indicator of a Worker's capacity. CPU and memory are weighted equally (30% each) as secondary indicators.
The weight validation ensures the formula is mathematically sound (percentages sum to 100):
Code evidence from `WorkerLoadBalancerConfigurationProperties.java:27`:
private WorkerLoadBalancerType type = WorkerLoadBalancerType.ROUND_ROBIN;
Dynamic weight defaults from `WorkerLoadBalancerConfigurationProperties.java:36-42`:
public static class DynamicWeightConfigProperties {
private int cpuUsageWeight = 30;
private int memoryUsageWeight = 30;
private int taskThreadPoolUsageWeight = 40;
Weight validation from `WorkerLoadBalancerConfigurationProperties.java:44-57`:
public void validated(Errors errors) {
if (cpuUsageWeight + memoryUsageWeight + taskThreadPoolUsageWeight != 100) {
errors.rejectValue("cpuUsageWeight", "cpuUsageWeight",
"cpuUsageWeight + memoryUsageWeight + threadUsageWeight must be 100");
}
}
Weight calculation from `DynamicWeightedRoundRobinWorkerLoadBalancer.java:70-76`:
private double calculateWeight(WorkerServerMetadata server) {
return 100 - (dynamicWeightConfigProperties.getCpuUsageWeight() * server.getCpuUsage()
+ dynamicWeightConfigProperties.getMemoryUsageWeight() * server.getMemoryUsage()
+ dynamicWeightConfigProperties.getTaskThreadPoolUsageWeight()
* server.getTaskThreadPoolUsage())
/ 3;
}