Workflow:Apache Kafka Release Candidate Staging
| Knowledge Sources | |
|---|---|
| Domains | Release_Engineering, Build_Automation |
| Last Updated | 2026-02-09 12:00 GMT |
Overview
End-to-end process for building, signing, and staging an Apache Kafka release candidate from the source repository through SVN and Maven artifact publication.
Description
This workflow automates the complete release candidate staging process for Apache Kafka. It covers prerequisite verification (GPG keys, JDK, Maven, SVN), version bumping across all project files, building source and binary tarballs, generating release notes, GPG-signing all artifacts with checksums, uploading to the Apache SVN dev staging area, and publishing Maven artifacts. The process produces a fully staged release candidate ready for community voting.
Usage
Execute this workflow when you are an Apache Kafka committer preparing a new release candidate for a version that has been approved for release. You need a configured GPG signing key, JDK 25, Apache Maven, SVN CLI access, and push access to both the Apache Kafka git repository and the Maven staging repository.
Execution Steps
Step 1: Verify Prerequisites
Validate that all required tools and credentials are available before beginning the release process. This includes checking for the GPG signing key and valid passphrase, verifying JDK 25 is installed and accessible, confirming Apache Maven CLI and SVN CLI are in PATH, and ensuring the local git working tree has no uncommitted changes.
Key considerations:
- GPG key ID and passphrase are read from ~/.gradle/gradle.properties
- The Apache ID is collected and persisted in a local preferences file
- All prerequisites are cached so re-runs skip already-verified checks
Step 2: Prepare Release Branch and Version Bump
Create a release branch from the development branch, update version numbers across all project files, and tag the release candidate. This involves modifying gradle.properties, kafkatest/__init__.py, version.py, and multiple Streams quickstart POM files to remove the -SNAPSHOT suffix.
Key considerations:
- A release branch (e.g., "3.7.0") is created from the remote dev branch (e.g., "3.7")
- The RC tag format is "{version}-rc{number}" (e.g., "3.7.0-rc0")
- The script registers cleanup hooks to delete branches and tags on failure
Step 3: Build Source and Binary Artifacts
Build the source tarball from the tagged RC, compile binary distribution tarballs using Gradle, and generate aggregated Javadoc documentation. The source archive is created directly from the git tag, while binary artifacts are built in a clean clone of the repository.
Key considerations:
- A fresh git clone is used for the build to ensure a clean environment
- The build uses Scala 2.13 exclusively
- Both source (kafka-{version}-src.tgz) and binary distributions are produced
- Javadoc is built with --no-parallel due to Gradle compatibility constraints
Step 4: Generate Release Notes
Automatically generate HTML-formatted release notes by querying JIRA for all issues resolved in this release version. The notes module formats bug fixes, improvements, new features, and other changes into a structured HTML document.
Key considerations:
- Release notes are written as RELEASE_NOTES.html in the artifacts directory
- The generation depends on JIRA API access for issue tracking data
Step 5: Sign and Checksum Artifacts
GPG-sign every artifact file and generate MD5, SHA1, and SHA512 checksums for verification. Each file in the artifacts directory receives a corresponding .asc signature file and hash files, enabling downstream verification of artifact integrity and authenticity.
Key considerations:
- Signing uses the GPG key configured in gradle.properties
- Each signature is verified immediately after creation
- Checksum commands use relative filenames to ensure portable verification
Step 6: Upload to SVN Staging
Package all signed artifacts into a compressed tarball and commit them to the Apache SVN development staging area. This makes the release candidate publicly available at the standard Apache dev distribution URL for community review and voting.
Key considerations:
- Artifacts are committed under the path corresponding to the RC tag
- The SVN dev URL follows the pattern https://dist.apache.org/repos/dist/dev/kafka/{rc_tag}
Step 7: Publish Maven Artifacts
Build and upload Maven artifacts to the Apache Maven staging repository, including both the main Kafka modules and the Streams quickstart archetype. This enables consumers to test the release candidate through standard Maven dependency resolution.
Key considerations:
- Main modules are published via Gradle's publish task with Scala 2.13
- The Streams quickstart is published separately via Maven with GPG signing profile
- After upload, the staging repository must be manually closed and verified
Step 8: Push Tags and Send Vote Email
Push the release candidate tag and the updated development branch to the remote Apache repository. Generate and display the RC vote email template for the committer to send to the dev mailing list, initiating the community vote.
Key considerations:
- The temporary release branch is deleted after pushing
- The vote email includes links to artifacts, source, Javadoc, and release notes
- The committer is prompted for manual verification steps before each push