Principle:Duckdb Duckdb Benchmark Build Configuration

Overview

Benchmark Build Configuration is the principle of enabling optional benchmark compilation targets within a build system. In large software projects, benchmark code is not part of the default build; instead, it is gated behind explicit build flags so that only developers or CI pipelines that need performance measurement pay the cost of compiling, linking, and shipping benchmark binaries. This principle ensures that production builds remain lean while still providing a clear, reproducible path to building benchmark tooling when needed.

Description

The core idea behind Benchmark Build Configuration is conditional compilation of benchmark code. Rather than always compiling benchmark targets (which may pull in additional dependencies, increase build times, and produce extra binaries), the build system exposes a feature flag that controls whether benchmark subdirectories and targets are included in the build graph.

Key aspects of this principle include:

• Build flag gating -- A boolean option (e.g., BUILD_BENCHMARKS) is declared in the build system configuration. When set to TRUE, the benchmark source directories are added to the build; when FALSE (the default), they are skipped entirely.
• Conditional directory inclusion -- The build system uses conditional logic to include or exclude entire subdirectories of benchmark code. This means that when benchmarks are disabled, no benchmark-related compilation or linking occurs.
• Dependency co-requirements -- Benchmarks may depend on other optional build components (such as unit test infrastructure). The build configuration must ensure that all prerequisite flags are also enabled when benchmarks are requested.
• Build target generation -- When enabled, the benchmark flag causes the build system to produce one or more executable targets (e.g., a benchmark_runner binary) that can be invoked to execute performance benchmarks.

Usage

Apply Benchmark Build Configuration when:

• Performance benchmarks need to be compiled alongside the main build, but should not be part of the default compilation.
• CI/CD pipelines need a deterministic, reproducible way to produce benchmark binaries for performance regression testing.
• Developers want to locally build and run benchmarks without modifying the core build configuration.
• The project maintains a separation between production artifacts and testing/benchmarking artifacts.

Theoretical Basis

This principle draws on several well-established software engineering concepts:

• Conditional compilation -- A technique dating back to C preprocessor #ifdef directives, extended in modern build systems (CMake, Bazel, Meson) to conditionally include entire source trees.
• Build-time feature flags -- The practice of exposing boolean or enumerated options at build configuration time that control which components are compiled. This is analogous to runtime feature flags but operates at the compilation stage.
• Separation of concerns in build systems -- Keeping benchmark code in dedicated subdirectories with their own build rules, included only when explicitly requested, follows the principle of modular build organization.
• Minimal default builds -- The idea that the default build configuration should produce the smallest, fastest-to-compile set of artifacts needed for normal development, with optional components added explicitly.

Related

• Implementation:Duckdb_Duckdb_Build_Benchmarks_Flag