Environment:Apache Dolphinscheduler ZooKeeper Registry
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Distributed_Systems |
| Last Updated | 2026-02-10 10:00 GMT |
Overview
Apache ZooKeeper 3.8.0 with Curator 5.5.0 client framework for cluster coordination, service registry, and distributed locking.
Description
DolphinScheduler uses a pluggable Registry abstraction for cluster coordination. The default and recommended implementation is Apache ZooKeeper 3.8.0 accessed through Apache Curator Framework 5.5.0. ZooKeeper provides service discovery (Master/Worker node registration), distributed locking (failover coordination), and ephemeral node heartbeats (failure detection). Alternative registry implementations exist for etcd (jetcd 0.5.11) and JDBC-based registry, but ZooKeeper is the production standard.
Usage
Use this environment for any production deployment of DolphinScheduler. ZooKeeper is required for Master-Worker cluster coordination, failover marker management, and workflow instance locking. Without a registry, DolphinScheduler can only run in standalone mode.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| ZooKeeper Server | 3.7.1+ (3.8.0 recommended) | Docker image: bitnami/zookeeper:3.7.1 |
| Network | Low-latency link to ZK ensemble | Session timeout default: 60s |
| Disk | SSD recommended for ZK data | ZK performance degrades on HDD |
| Ensemble Size | 3+ nodes for production | Odd numbers for quorum (3, 5, 7) |
Dependencies
System Packages
- `zookeeper` = 3.8.0 (server-side)
Java Dependencies (client-side, managed by Maven BOM)
- `curator-framework` = 5.5.0
- `curator-recipes` = 5.5.0
- `zookeeper` = 3.8.0 (client JAR)
- `dropwizard-metrics` = 4.2.11
- `snappy-java` (compression)
Alternative Registry Dependencies
- `jetcd-core` = 0.5.11 (for etcd registry)
- `jetcd-test` = 0.7.1 (for etcd test)
Credentials
The following configuration properties must be set in `application.yaml`:
- `registry.type`: Registry type (`zookeeper`, `etcd`, or `jdbc`)
- `registry.zookeeper.connect-string`: ZK connection string (e.g., `localhost:2181`)
- `registry.zookeeper.namespace`: ZK namespace (default: `dolphinscheduler`)
If ZooKeeper authentication is enabled:
- `registry.zookeeper.digest`: ZK digest authentication string
Quick Install
# Run ZooKeeper via Docker (development)
docker run -d --name zookeeper \
-p 2181:2181 \
-e ALLOW_ANONYMOUS_LOGIN=yes \
-e ZOO_4LW_COMMANDS_WHITELIST=srvr,ruok,wchs,cons \
bitnami/zookeeper:3.7.1
# Verify ZooKeeper is running
echo ruok | nc localhost 2181
Code Evidence
Default registry configuration from `dolphinscheduler-master/src/main/resources/application.yaml`:
registry:
type: zookeeper
zookeeper:
namespace: dolphinscheduler
connect-string: localhost:2181
retry-policy:
base-sleep-time: 1s
max-sleep: 3s
max-retries: 5
session-timeout: 60s
connection-timeout: 15s
block-until-connected: 15s
ZooKeeper version from `dolphinscheduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-zookeeper/pom.xml`:
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
Docker Compose deployment from `deploy/docker/docker-compose.yml:39-53`:
dolphinscheduler-zookeeper:
image: bitnami/zookeeper:3.7.1
environment:
ALLOW_ANONYMOUS_LOGIN: "yes"
ZOO_4LW_COMMANDS_WHITELIST: srvr,ruok,wchs,cons
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `KeeperErrorCode = ConnectionLoss` | ZK server unreachable | Verify ZK is running and `connect-string` is correct |
| `Session expired` | ZK session timeout exceeded | Increase `session-timeout` or reduce GC pauses |
| `NodeExists for /dolphinscheduler` | Stale ZK data from previous deployment | Clean namespace: `deleteall /dolphinscheduler` in zkCli |
| `Unable to create /registry/master` | Missing ZK namespace | Ensure namespace is created or set `namespace: dolphinscheduler` |
Compatibility Notes
- ZooKeeper 3.4.x: Supported via the `zk-3.4` Maven profile which uses Curator 4.3.0 and ZooKeeper 3.4.14. Not recommended for new deployments.
- etcd: Alternative registry via `jetcd-core` 0.5.11. Requires separate etcd cluster deployment.
- JDBC Registry: Uses the existing database as registry (no external dependency). Suitable for small deployments but has higher latency than ZooKeeper.