Implementation:Apache Spark Release Build Publish

Knowledge Sources	Apache Spark
Domains	Release_Engineering
Type	API Doc
Last Updated	2026-02-08 12:00 GMT

Overview

Release build script sub-command that uploads signed Maven artifacts to Apache Nexus staging and sends RC vote emails.

Description

release-build.sh publish-release creates a Nexus staging repository, uploads all Maven artifacts with GPG signatures using parallel xargs, closes the staging repository, and sends an RC vote email to dev@spark.apache.org. It handles both Scala 2.12 and 2.13 artifact sets and uses the Apache Nexus REST API for repository management.

The publishing process involves the following operations:

1. Create staging repository: Uses the Nexus REST API to create a new staging repository under the Apache Spark staging profile.
2. Upload Maven artifacts: Iterates through all built JARs, POMs, source JARs, and javadoc JARs, uploading each with its GPG signature to the staging repository. Uploads are parallelized using xargs for performance.
3. Close staging repository: Closes the staging repository via the Nexus REST API, which triggers Nexus validation rules (signature verification, POM completeness, etc.).
4. Upload binary distributions: Uploads binary tarballs, signatures, and checksums to the Apache SVN dev area.
5. Send vote email: Composes and sends an RC vote email to the Spark development mailing list with links to all artifacts and verification instructions.

The script uses the Apache Nexus REST API at https://repository.apache.org/service/local/staging with the staging profile ID d63f592e7eac0.

Usage

Run after the package step to upload artifacts for community voting. This step makes the release candidate available for public verification.

Code Reference

Source Location

• Repository: apache/spark
• File: dev/create-release/release-build.sh (lines 901-1095)

Signature

dev/create-release/release-build.sh publish-release

Key Constants

Constant	Value	Description
NEXUS_ROOT	https://repository.apache.org/service/local/staging	Base URL for the Nexus staging REST API
NEXUS_PROFILE	d63f592e7eac0	Apache Spark staging profile ID in Nexus

Required Environment Variables

Variable	Description
ASF_USERNAME	Apache Software Foundation username for Nexus authentication
ASF_NEXUS_TOKEN	Nexus API token for artifact uploads
GPG_PASSPHRASE	Passphrase for GPG signing during Maven deployment
PUBLISH_PROFILES	Maven profiles to activate during artifact publishing

I/O Contract

Inputs

Name	Type	Required	Description
Built artifacts	JARs, POMs	Yes	Maven artifacts produced by the package step
GPG key	GPG keyring	Yes	Release manager's key for signing Maven artifacts during upload
ASF Nexus credentials	environment variables	Yes	ASF_USERNAME, ASF_NEXUS_TOKEN for Nexus API access
Maven profiles	environment variable	Yes	PUBLISH_PROFILES specifying which profiles to publish

Outputs

Name	Type	Description
Nexus staging repository	Nexus repo	Created and closed staging repository containing all Maven artifacts
SVN artifacts	SVN directory	Binary distributions uploaded to Apache dev SVN
RC vote email	email	Vote email sent to dev@spark.apache.org with artifact links

Usage Examples

Publish Release

# Publish release artifacts (runs with env vars set by do-release-docker.sh)
dev/create-release/release-build.sh publish-release

Nexus API Interactions

# The script internally performs these Nexus REST API calls:

# 1. Create staging repository
curl -u $ASF_USERNAME:$ASF_NEXUS_TOKEN \
  -H "Content-Type:application/json" \
  -d '{"data":{"description":"Apache Spark 3.5.0-rc1"}}' \
  "$NEXUS_ROOT/profiles/$NEXUS_PROFILE/start"

# 2. Upload artifacts (parallelized via xargs)
# Each artifact is uploaded to:
#   $NEXUS_ROOT/deployByRepositoryId/$STAGING_REPO_ID/...

# 3. Close staging repository
curl -u $ASF_USERNAME:$ASF_NEXUS_TOKEN \
  -H "Content-Type:application/json" \
  -d '{"data":{"stagedRepositoryId":"$STAGING_REPO_ID"}}' \
  "$NEXUS_ROOT/profiles/$NEXUS_PROFILE/finish"

Related Pages

Implements Principle

• Principle:Apache_Spark_Release_Artifact_Publishing
• Environment:Apache_Spark_Release_Build_Environment