Environment:Apache Druid Druid Operator Kubernetes

Knowledge Sources	Apache Druid
Domains	Kubernetes, Go, Operator Pattern, Helm, CRD
Last Updated	2026-02-10 10:00 GMT

Overview

The Druid Operator Kubernetes environment provides the Go-based Kubernetes operator that provisions, manages, and scales Apache Druid clusters on Kubernetes using custom resource definitions (CRDs).

Description

The Apache Druid Kubernetes Operator is a Go 1.20 application built with Kubebuilder v3 and the controller-runtime framework (v0.15.3). It manages Druid clusters through two custom resources: Druid and DruidIngestion, both belonging to the druid.apache.org API group at version v1alpha1.

The operator watches for changes to Druid custom resources and reconciles the desired state by creating and managing Kubernetes StatefulSets, Services, ConfigMaps, and PersistentVolumeClaims for each Druid node type (coordinator, overlord, broker, historical, middle manager, router). It supports distributed mode only and is designed for production Druid deployments on Kubernetes.

The project includes a Helm chart for installation, Kustomize configurations for CRD management, and an end-to-end test suite that uses Kind (Kubernetes in Docker) with a local registry, MinIO for deep storage, and ZooKeeper for coordination.

Development tooling includes controller-gen for CRD/RBAC generation, setup-envtest for controller unit tests against a local Kubernetes API server, and gen-crd-api-reference-docs for API documentation generation.

Usage

Use this environment when:

• Deploying and managing Apache Druid clusters on Kubernetes
• Developing new features or fixing bugs in the Druid operator
• Running operator unit tests or end-to-end tests
• Building and publishing the operator Docker image
• Generating or updating CRD manifests and Helm charts

System Requirements

Category	Requirement	Notes
Go	1.20	go.mod:20
Kubernetes	1.22 - 1.33.1	See compatibility matrix; <=1.20 not supported, 1.21 only with operator 0.0.9
ENVTEST K8s	1.26.0	Makefile:37: ENVTEST_K8S_VERSION = 1.26.0
Docker	Required	For building operator images
Helm	v3	For chart installation and templating
Kustomize	v3.8.7	Makefile:249; downloaded automatically
controller-gen	v0.14.0	Makefile:250; downloaded automatically
kubectl	Compatible with target cluster	For CRD installation and operator deployment

Dependencies

System Packages

• Go 1.20
• Docker (for building and pushing operator images)
• kubectl (for cluster interaction)
• Helm v3 (for chart-based installation)
• Kind (for local E2E testing)
• Bash (for E2E test scripts)

Language Packages

Direct Go dependencies (from go.mod):

• k8s.io/api v0.27.7
• k8s.io/apimachinery v0.27.7
• k8s.io/client-go v0.27.7
• sigs.k8s.io/controller-runtime v0.15.3
• github.com/datainfrahq/operator-runtime v0.0.2
• github.com/ghodss/yaml v1.0.0
• github.com/go-logr/logr v1.2.4
• github.com/onsi/ginkgo/v2 v2.9.5 (testing)
• github.com/onsi/gomega v1.27.7 (testing)
• github.com/stretchr/testify v1.8.1 (testing)

Build tool versions (from Makefile):

• Kustomize v3.8.7
• controller-tools v0.14.0
• gen-crd-api-reference-docs v0.3.0

Credentials

No secrets are required for development. The following variables are used in the build and test process:

Variable	Default	Purpose
IMG	datainfrahq/druid-operator	Docker image repository for the operator
IMG_TAG	latest	Docker image tag
IMG_KIND	localhost:5001/druid-operator	Image URL for Kind local registry
NAMESPACE_DRUID_OPERATOR	druid-operator-system	Kubernetes namespace for the operator
NAMESPACE_DRUID	druid	Kubernetes namespace for the Druid application
NAMESPACE_ZK_OPERATOR	zk-operator	Kubernetes namespace for the ZooKeeper operator
NAMESPACE_MINIO_OPERATOR	minio-operator	Kubernetes namespace for the MinIO operator
MINIO_VERSION	6.0.4	MinIO Helm chart version for E2E tests

Quick Install

# Clone the repository
git clone https://github.com/apache/druid.git
cd druid/druid-operator

# Generate CRDs and code
make manifests generate

# Run unit tests
make test

# Build the operator binary
make build

# Build the Docker image
make docker-build IMG=myregistry/druid-operator:latest

# Install CRDs into the cluster
make install

# Deploy the operator via Kustomize
make deploy IMG=myregistry/druid-operator:latest

# Or install via Helm
helm upgrade --install \
  --namespace druid-operator-system \
  --create-namespace \
  druid-operator-system chart/ \
  --set image.repository=myregistry/druid-operator \
  --set image.tag=latest

# Run E2E tests (requires Kind)
make kind
make e2e

# Uninstall
make undeploy
make uninstall

Code Evidence

Go version (druid-operator/go.mod:20):

go 1.20

K8s client library versions (druid-operator/go.mod:29-32):

require (
	k8s.io/api v0.27.7
	k8s.io/apimachinery v0.27.7
	k8s.io/client-go v0.27.7
	sigs.k8s.io/controller-runtime v0.15.3
)

ENVTEST Kubernetes version (druid-operator/Makefile:37):

ENVTEST_K8S_VERSION = 1.26.0

Tool versions (druid-operator/Makefile:248-251):

## Tool Versions
KUSTOMIZE_VERSION ?= v3.8.7
CONTROLLER_TOOLS_VERSION ?= v0.14.0
GEN_CRD_API_REF_VERSION ?= v0.3.0

CRD generation command (druid-operator/Makefile:78-80):

manifests: controller-gen
	$(CONTROLLER_GEN) crd:generateEmbeddedObjectMeta=true rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
	$(CONTROLLER_GEN) crd:generateEmbeddedObjectMeta=true paths="./..." output:crd:artifacts:config=chart/crds/

Unit test with envtest (druid-operator/Makefile:96-97):

test: manifests generate fmt vet envtest
	KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" go test ./... -coverprofile cover.out

Multi-platform Docker build support (druid-operator/Makefile:182):

PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le

Kubernetes version compatibility matrix (from druid-operator/README.md:66-71):

druid-operator	0.0.9	v1.0.0	v1.1.0+
kubernetes <= 1.20	Not supported	Not supported	Not supported
kubernetes == 1.21	Supported	Not supported	Not supported
kubernetes 1.22-1.25	Supported	Supported	Supported
kubernetes > 1.25 to 1.33.1	Not supported	Not supported	Supported

Common Errors

Error Message	Cause	Solution
go: go.mod requires go >= 1.20	Go version older than 1.20	Install Go 1.20 or newer
controller-gen: command not found	Build tools not yet downloaded	Run make controller-gen to download the tool
KUBEBUILDER_ASSETS not set	envtest binary not installed	Run make envtest to install setup-envtest
no matches for kind "Druid" in version "druid.apache.org/v1alpha1"	CRDs not installed in the cluster	Run make install to apply CRDs
Helm template error: unknown field in Druid spec	CRD version mismatch between chart and cluster	Regenerate CRDs with make helm-generate and reapply
E2E test failure: connection refused to Kind cluster	Kind cluster not running or registry not accessible	Run make kind to bootstrap the Kind cluster with registry
HPA errors after upgrade	HPA API changed from autoscaling/v2beta1 to autoscaling/v2	Update HPA spec in Druid CR to use autoscaling/v2 syntax

Compatibility Notes

• The operator moved to Kubebuilder v3, which requires a manual migration step for users upgrading from earlier versions (see docs/kubebuilder_v3_migration.md).
• Kubernetes <= 1.20 is not supported by any version of the operator.
• Kubernetes 1.21 is only supported by operator version 0.0.9.
• Kubernetes > 1.25 requires operator version v1.1.0 or later.
• The operator moved from Ingress apiVersion networking/v1beta1 to networking/v1. Users must update their Ingress spec in the Druid CR accordingly.
• The operator moved from HPA apiVersion autoscaling/v2beta1 to autoscaling/v2. Users must update their HPA specs.
• Release v1.2.2 had a bug for namespace-scoped operator deployments; this was fixed in v1.2.3.
• Multi-platform Docker builds support linux/arm64, linux/amd64, linux/s390x, and linux/ppc64le via Docker buildx.
• The operator provisions Druid in distributed mode only; single-server deployments are not supported.
• The operator is separate from the web console codebase but lives in the same repository under druid-operator/.

Related Pages

No direct implementation page dependencies (the operator is separate from the web console).