Implementation:Apache Druid Integration Test Runner

Knowledge Sources	Apache_Druid
Domains	Testing, CI_CD, Infrastructure
Last Updated	2026-02-10 10:00 GMT

Overview

it.sh is a Bash utility script that wraps complex Maven commands to simplify running Apache Druid integration tests both locally and in CI pipelines.

Description

This script provides a command-line interface for the full lifecycle of Druid integration tests: building the distribution, constructing Docker test images, starting and stopping Docker Compose clusters, running test suites by JUnit category, and tailing container logs. It bridges environment variables from the host into Docker containers through an override file mechanism, supporting credentials and configuration from local files, environment variables, and CI systems such as GitHub Actions.

Usage

Use this script when running Druid integration tests locally during development or when configuring CI pipelines. It replaces lengthy Maven invocations with short, memorable subcommands.

Code Reference

Source Location

• Repository: Apache Druid
• File: it.sh
• Lines: 1-316

Signature

# Top-level command dispatch
# Usage: it.sh cmd [category] [module]
#
# Commands:
#   ci          - Build Druid and distribution for CI pipelines
#   build       - Build Druid and the distribution
#   dist        - Build the Druid distribution only
#   tools       - Build druid-it-tools
#   image       - Build the test Docker image
#   up          - Start the cluster for a test category
#   down        - Stop the cluster for a test category
#   run         - Run tests on an already-running cluster
#   test        - Start cluster, run tests, stop cluster
#   tail        - Show last 100 lines of each container log
#   gen         - Generate docker-compose.yaml files
#   github      - Run one IT in GitHub Workflows (test + tail on failure)
#   prune-containers - Stop all running Docker containers
#   prune-volumes    - Prune Docker volumes

# Key internal functions:
function usage { ... }
function tail_logs { ... }
function build_override { ... }
function reuse_override { ... }
function require_category { ... }
function require_env_var { ... }
function verify_env_vars { ... }

Import

# Run from the repository root:
./it.sh <command> [category] [module]

I/O Contract

Inputs

Name	Type	Required	Description
cmd	string	Yes	The subcommand to execute (ci, build, dist, tools, image, up, down, run, test, tail, gen, github, prune-containers, prune-volumes)
category	string	Conditional	JUnit test category name (required for up, down, run, test, tail, gen, github)
module	string	No	Relative path to the Maven module with tests; defaults to integration-tests-ex/cases
OVERRIDE_ENV	env var	No	Path to an environment override file with var=value pairs passed to Docker containers
USE_INDEXER	env var	No	Set to "middleManager" (default) or "indexer" to select the indexing service type
druid_*	env var	No	Any environment variable prefixed with druid_ is forwarded into Docker containers

Outputs

Name	Type	Description
exit code	integer	0 on success, non-zero on failure (Maven test result is propagated)
override.env	file	Generated environment override file at <module>/target/override.env used by Docker Compose
container logs	stdout	When using the tail or github commands, container log tails are printed to stdout

Usage Examples

Build Distribution and Run an Integration Test

# Full build for CI
./it.sh ci

# Build Druid distribution and test image
./it.sh build
./it.sh image

# Run the BatchIndex integration test category end-to-end
./it.sh test BatchIndex

# Run a test from a specific module
./it.sh test BatchIndex integration-tests-ex/cases

GitHub Actions Workflow

# In GitHub Actions, run a test category and auto-tail logs on failure
./it.sh github BatchIndex

Start a Cluster for Debugging

# Start the cluster without running tests
./it.sh up BatchIndex

# Manually run tests against the running cluster
./it.sh run BatchIndex

# Inspect logs
./it.sh tail BatchIndex

# Tear down
./it.sh down BatchIndex