Implementation:Vespa engine Vespa Build Rpms Sh

CI_CD Build_Systems

Overview

This page documents the implementation of the Vespa RPM package creation script: .buildkite/build-rpms.sh. This script generates a source RPM, rebuilds it into binary RPMs with zstd compression, and creates a YUM repository from the results.

Type: External Tool Doc

Code Reference

#!/usr/bin/env bash
# .buildkite/build-rpms.sh (L1-30)

set -o errexit
set -o nounset
set -o pipefail

if [[ -n "${DEBUG:-}" ]]; then
    set -o xtrace
fi

echo "--- Building RPM packages"
ulimit -c 0

echo "Creating source RPM..."
make  -f .copr/Makefile srpm outdir="$WORKDIR"

echo "Building binary RPMs..."
rpmbuild --rebuild \
  --define="_topdir $WORKDIR/vespa-rpmbuild" \
  --define "_debugsource_template %{nil}" \
  --define "_binary_payload w10T4.zstdio" \
  --define "installdir $WORKDIR/vespa-install" "$WORKDIR"/vespa-"$VESPA_VERSION"-*.src.rpm

echo "Moving RPMs and creating repository..."
mv "$WORKDIR"/vespa-rpmbuild/RPMS/*/*.rpm "$WORKDIR/artifacts/$ARCH/rpms"
createrepo "$WORKDIR/artifacts/$ARCH/rpms"

I/O Contract

Inputs (Environment Variables)

Variable	Required	Description	Example
WORKDIR	Yes	Working directory for all build artifacts	/tmp/vespa-build
VESPA_VERSION	Yes	Version string for the RPM package name	8.432.17
ARCH	Yes	CPU architecture identifier	x86_64 or aarch64
DEBUG	No	If set to a non-empty value, enables bash xtrace	1

Inputs (Prerequisite Files)

File/Directory	Produced By	Description
.copr/Makefile	Source repository	Makefile with srpm target for SRPM generation
$WORKDIR/vespa-install/	make install (C++ build)	Pre-built binaries and libraries
$WORKDIR/artifacts/$ARCH/rpms/	prepare.sh	Pre-created empty directory for RPM output

Outputs (Files)

Output	Type	Description
$WORKDIR/vespa-$VESPA_VERSION-*.src.rpm	Source RPM	Contains spec file and source tarball
$WORKDIR/artifacts/$ARCH/rpms/*.rpm	Binary RPMs	Installable RPM packages
$WORKDIR/artifacts/$ARCH/rpms/repodata/	YUM metadata	Repository metadata for yum/dnf

Key Implementation Details

Core Dump Suppression

ulimit -c 0

The script sets the core dump size limit to zero at the outset. This prevents any process that crashes during the RPM build from writing a core dump file, which could consume gigabytes of disk space and potentially fill the build disk.

Source RPM Generation

make -f .copr/Makefile srpm outdir="$WORKDIR"

The .copr/Makefile follows the Copr build service convention. The srpm target:

1. Generates a source tarball from the current source tree.
2. Combines the tarball with the RPM spec file (vespa.spec).
3. Produces an SRPM file named vespa-$VESPA_VERSION-*.src.rpm in $WORKDIR.

The outdir variable is passed to the Makefile to control where the SRPM is written.

Binary RPM Rebuild

rpmbuild --rebuild \
  --define="_topdir $WORKDIR/vespa-rpmbuild" \
  --define "_debugsource_template %{nil}" \
  --define "_binary_payload w10T4.zstdio" \
  --define "installdir $WORKDIR/vespa-install" \
  "$WORKDIR"/vespa-"$VESPA_VERSION"-*.src.rpm

The rpmbuild --rebuild command processes the SRPM through the full RPM build lifecycle. The --define flags override default RPM macros:

• _topdir: Redirects the RPM build tree (BUILD, RPMS, SRPMS, SOURCES, SPECS directories) to an isolated working directory. This avoids conflicts with any system-level RPM build configuration.
• _debugsource_template %{nil}: Sets the debug source template to nil, suppressing the creation of debugsource RPMs. This reduces build time and disk usage.
• _binary_payload w10T4.zstdio: Configures the RPM payload compression:
  • w: Write mode
  • 10: Compression level 10 (high compression)
  • T4: Use 4 threads for compression
  • zstdio: Use the Zstandard algorithm
• installdir: Points to the directory where make install placed the compiled binaries. The spec file uses this to avoid recompilation during the %install phase.

The glob pattern vespa-"$VESPA_VERSION"-*.src.rpm matches the SRPM file regardless of the release suffix.

RPM Collection and Repository Creation

mv "$WORKDIR"/vespa-rpmbuild/RPMS/*/*.rpm "$WORKDIR/artifacts/$ARCH/rpms"
createrepo "$WORKDIR/artifacts/$ARCH/rpms"

After rpmbuild completes, the binary RPMs are scattered across architecture-specific subdirectories under RPMS/ (e.g., RPMS/x86_64/, RPMS/noarch/). The mv command with the glob */*.rpm collects all RPMs into the flat artifact directory.

The createrepo command then generates YUM repository metadata, creating a repodata/ directory containing:

• repomd.xml: Repository metadata index
• primary.xml.gz: Package names, versions, and dependencies
• filelists.xml.gz: File lists for each package
• other.xml.gz: Changelog and supplementary data

This enables downstream consumers (the container image build and artifact publishing) to use the directory as a YUM repository.

Execution Context

Buildkite Pipeline
  --> .buildkite/prepare.sh (creates artifact directories)
  --> .buildkite/bootstrap.sh + java.sh (Java build)
  --> .buildkite/bootstrap-cmake.sh + cpp.sh (C++ build + make install)
  --> .buildkite/build-rpms.sh
        --> ulimit -c 0
        --> make -f .copr/Makefile srpm
        --> rpmbuild --rebuild ...
        --> mv RPMs to artifact directory
        --> createrepo

Source File Locations

• .buildkite/build-rpms.sh (Lines 1-30)

See Also

• RPM Package Creation Principle -- The design rationale for RPM packaging in the Vespa build.
• C++ Compilation Implementation -- The preceding stage that produces compiled binaries.
• Container Image Building Implementation -- The next stage that consumes RPMs.
• Artifact Publishing Implementation -- The stage that signs and uploads RPMs.

Related Pages

• Principle:Vespa_engine_Vespa_RPM_Package_Creation
• Environment:Vespa_engine_Vespa_CMake_Cpp23_Build_Environment
• Heuristic:Vespa_engine_Vespa_Maven_Parallel_Build_Optimization
• Heuristic:Vespa_engine_Vespa_RPM_Zstd_Compression_Settings