Principle:Duckdb Duckdb Extension Build And Metadata
| Field | Value |
|---|---|
| sources | extension/extension_build_tools.cmake, scripts/append_metadata.cmake, extension/generated_extension_loader.cpp.in
|
| domains | Extension_Development, Build_System |
| last_updated | 2026-02-07 |
Overview
Compiling extension modules into self-describing binary artifacts with embedded metadata. This principle governs the process by which DuckDB loadable extension shared libraries (.duckdb_extension files) are built with appended metadata sections that encode platform, version, ABI type, and a signature placeholder directly into the binary artifact.
Description
When a DuckDB extension is compiled as a loadable module, the resulting shared library must carry enough information for the runtime to verify compatibility and authenticity without requiring any external manifest or sidecar file. This is achieved by appending a structured metadata section to the end of the compiled binary.
The metadata section contains:
- Platform identifier -- the target platform string (e.g.,
linux_amd64,osx_arm64,windows_amd64) - DuckDB version -- the version of DuckDB the extension was built against
- Extension version -- the version of the extension itself
- ABI type -- the Application Binary Interface type (e.g.,
C_STRUCTorCPP), which determines binary compatibility rules - Signature placeholder -- a 256-byte region reserved for a cryptographic signature to be appended during the signing step
This approach makes each .duckdb_extension file a self-describing artifact. The DuckDB runtime can read the metadata from a loaded binary and perform compatibility checks before executing any extension code. If the platform or ABI type does not match, the extension is rejected at load time, preventing crashes or undefined behavior from incompatible binaries.
Why metadata matters for distribution and verification:
- Compatibility checking -- the runtime can verify at load time that the extension was built for the correct platform and DuckDB version without relying on filename conventions or external metadata files.
- Signature verification -- the reserved signature region enables a cryptographic signing workflow where the binary is signed after compilation, and the runtime can verify the signature before loading.
- Versioning -- embedding version information allows the runtime to detect stale or mismatched extensions and provide meaningful error messages.
- Self-containment -- a single
.duckdb_extensionfile contains everything needed for validation, eliminating the need for companion metadata files during distribution.
Usage
This principle applies when developing extensions that will be distributed as loadable modules. It is the first step in the Extension Development and Distribution workflow, preceding signing, compression, upload, and verification.
Typical scenarios include:
- Building an in-tree DuckDB extension (e.g.,
httpfs,parquet,json) as part of the standard CMake build - Building an out-of-tree extension using the DuckDB extension template
- Creating platform-specific builds in CI/CD pipelines for distribution across multiple operating systems and architectures
Theoretical Basis
- Binary metadata sections -- many binary formats (ELF, Mach-O, PE) support custom sections or appended data. DuckDB uses a simpler approach of appending fixed-format metadata to the end of the shared library, which is portable across all platforms.
- Self-describing artifacts -- inspired by practices in container image distribution (OCI manifests) and package management systems where artifacts carry their own metadata for validation.
- ABI compatibility -- the ABI type field ensures that extensions compiled with one ABI convention are not loaded into a runtime expecting another, preventing memory layout mismatches and undefined behavior at the C/C++ boundary.