Workflow:Apache Kafka Docker Image Release
| Knowledge Sources | |
|---|---|
| Domains | Release_Engineering, Containerization |
| Last Updated | 2026-02-09 12:00 GMT |
Overview
End-to-end process for building, testing, and publishing multi-architecture Docker images for Apache Kafka release candidates and official releases.
Description
This workflow documents the Docker image release pipeline for Apache Kafka, covering local build testing, multi-architecture (amd64/arm64) image construction using Docker Buildx, and publication to container registries. It includes the standard JVM-based image type and the native image variant. The process also covers the Docker Official Image pipeline which prepares source artifacts for the official Docker library repository. The workflow ensures that Docker images are properly tested before publication and tagged with correct metadata.
Usage
Execute this workflow when preparing Docker images for a Kafka release candidate or promoting to a final release. You need Docker with Buildx support enabled, login credentials for the target container registry, and access to the Kafka binary distribution tarball URL.
Execution Steps
Step 1: Build and Test Locally
Build a Docker image locally using the docker_build_test.py script to validate the Dockerfile and image contents before any remote publication. The script builds the image for the local architecture using a specified Kafka binary URL and image type (JVM or native), then runs basic sanity checks.
Key considerations:
- The build test uses the standard Docker build (not Buildx) for local validation
- Image type can be "jvm" (default) or "native"
- The Kafka binary URL points to the release tarball to be included in the image
- The build_date argument is set automatically to the current date
Step 2: Create Buildx Builder
Initialize a Docker Buildx builder instance named "kafka-builder" to support multi-architecture image builds. The builder is configured to support cross-platform compilation for both linux/amd64 and linux/arm64 targets.
Key considerations:
- The builder is created using "docker buildx create --name kafka-builder --use"
- Buildx must be available in the Docker installation
- The builder is removed automatically after the build completes (or on failure)
Step 3: Build Multi-Architecture Image
Build the Kafka Docker image for both amd64 and arm64 architectures simultaneously using Docker Buildx. The build uses the appropriate Dockerfile based on image type and injects the Kafka binary URL and build date as build arguments.
Key considerations:
- Platform targets are linux/amd64 and linux/arm64
- The Dockerfile is selected based on image type (JVM vs native)
- Build arguments include kafka_url and build_date
- The --push flag combines build and push into a single step
Step 4: Push to Container Registry
Push the multi-architecture image manifest and platform-specific layers to the target container registry. The image tag should follow the format registry/namespace/image_name:image_tag (e.g., apache/kafka:3.7.0-rc0).
Key considerations:
- Ensure you are logged into the target registry before pushing
- The image manifest includes both architecture variants
- Image sanity testing should be completed before pushing
Step 5: Clean Up Builder
Remove the Buildx builder instance created for the multi-architecture build. This cleanup runs in a finally block to ensure the builder is removed even if the build or push fails.
Key considerations:
- The builder is removed using "docker buildx rm kafka-builder"
- Cleanup is automatic via try/finally in the script
- Failure to clean up leaves the builder available for debugging