Principle:ClickHouse ClickHouse Nfpm Package Generation

Knowledge Sources	ClickHouse
Domains	Packaging, Distribution
Last Updated	2026-02-08 00:00 GMT

Overview

Nfpm package generation is the practice of creating Linux distribution packages (DEB and RPM) from compiled binaries using declarative YAML-based package definitions rather than traditional packaging toolchains.

Description

nfpm (Not FPM) is a Go-based tool that reads YAML configuration files describing a package's metadata, dependencies, file mappings, and scripts, then produces ready-to-install DEB or RPM packages. It replaces the traditional approach of using dpkg-deb for Debian packages and rpmbuild for Red Hat packages, which each require their own directory structures, control files, and spec files.

In the ClickHouse project, nfpm is used to generate all distribution packages from a unified set of YAML specifications stored in the packages/ directory. Each YAML file describes one package: its name, version, architecture, dependencies, file contents, symlinks, configuration files, and post-install scripts. The same YAML file is used to generate both DEB and RPM variants by running nfpm with different -p flags.

The YAML specifications use environment variable interpolation (e.g., ${CLICKHOUSE_VERSION_STRING}, ${DEB_ARCH}) to inject build-time values, and support format-specific overrides (e.g., different dependency syntax for DEB vs RPM). Package sources -- the actual files to include -- are placed in a root/ subdirectory mirroring the target filesystem layout.

Usage

Nfpm package generation should be used when:

• Building official ClickHouse releases for distribution via APT and YUM repositories.
• Creating CI/CD artifacts that need to be installable on Debian-based and Red Hat-based Linux distributions.
• Testing package installation in integration tests that simulate real deployment scenarios.
• Adding new packages to the ClickHouse distribution (by creating a new YAML file following the established pattern).

Theoretical Basis

Traditional vs Declarative Packaging

Traditional Linux packaging requires separate toolchains and directory structures for each target format:

• Debian (dpkg-deb): Requires a DEBIAN/ directory with control, conffiles, postinst, and other control files, plus a filesystem tree mirroring the target layout.
• RPM (rpmbuild): Requires a .spec file with macros, build instructions, file lists, and scriptlets, operating within a predefined BUILD/RPMS/SOURCES/SPECS/SRPMS directory hierarchy.

Maintaining parallel definitions for both formats introduces duplication and drift risk. Declarative packaging tools like nfpm solve this by providing a single source of truth (the YAML file) from which both formats are generated.

YAML Package Specification Structure

A typical nfpm YAML specification contains:

• Metadata: name, version, arch, platform, description, vendor, license, homepage, maintainer.
• Relationships: depends, recommends, suggests, replaces, conflicts, provides.
• Contents: A list of source-to-destination file mappings. Each entry specifies src (local path), dst (installed path), and optionally type (e.g., config|noreplace for configuration files, symlink for symbolic links).
• Scripts: Hooks for preinstall, postinstall, preremove, and postremove actions.
• Format overrides: The overrides section allows format-specific fields, such as different dependency syntax for DEB (package (= version)) vs RPM (package = version).

Environment Variable Interpolation

nfpm supports shell-style environment variable expansion (${VAR}) throughout the YAML file. ClickHouse leverages this to inject:

• ${CLICKHOUSE_VERSION_STRING} -- the full version string (e.g., 24.3.1.1).
• ${DEB_ARCH} -- the target architecture (e.g., amd64, arm64).

This allows the same YAML files to be reused across versions and architectures without modification.

Advantages Over Traditional Toolchains

• Single source of truth: One YAML file produces both DEB and RPM packages.
• Simplicity: No need to learn Debian control file syntax or RPM spec file macros.
• Portability: nfpm is a single Go binary with no system dependencies.
• Reproducibility: YAML files are easy to version-control and diff.
• CI integration: nfpm can be invoked as a single command in CI pipelines without complex build environments.

Related Pages

Implemented By

• Implementation:ClickHouse_ClickHouse_Nfpm_Common_Static_Package