Principle:Duckdb Duckdb Extension Loading Verification

Field	Value
sources	scripts/extension-upload-test.sh, scripts/run_extension_medata_tests.sh
domains	Extension_Development, Testing
last_updated	2026-02-07

Overview

Validating that distributed extensions can be successfully installed and loaded at runtime. This principle governs the end-to-end testing of the extension distribution pipeline, ensuring that extensions uploaded to cloud storage can be downloaded, decompressed, verified, and loaded by the DuckDB runtime without errors.

Description

After extensions have been built, signed, compressed, and uploaded to cloud storage, there is a critical need to verify the entire distribution pipeline before making extensions available to end users. Extension Loading Verification provides this guarantee through automated smoke tests that exercise the full install-and-load path.

The verification process tests the following:

1. Download and decompression -- the DuckDB runtime can successfully download the compressed extension from the S3 URL and decompress it
2. Metadata validation -- the embedded metadata (platform, version, ABI type) matches the current DuckDB runtime, and the metadata is structurally valid
3. Signature verification -- the cryptographic signature is valid, confirming the extension has not been tampered with since signing
4. Loading and initialization -- the extension's shared library can be loaded into the process and its initialization function executes successfully
5. Basic functionality -- a simple query using the extension's functionality returns expected results, confirming the extension is not just loadable but operational

The verification uses the standard SQL INSTALL and LOAD commands:

INSTALL 'httpfs';
LOAD 'httpfs';
-- Optionally run a basic functional test query

Why verification matters:

• A failure at any step in the build-sign-compress-upload pipeline could produce a binary that appears to be distributed correctly (the file exists at the expected URL) but fails at runtime. Examples include:
  • Incorrect metadata appended due to a CMake configuration error
  • Signature computed over the wrong binary content
  • Compression corruption during upload
  • Platform mismatch between the build environment and the declared platform string
• Catching these failures before promoting to production prevents broken extensions from reaching end users
• Verification serves as an integration test for the entire distribution pipeline, complementing the unit tests of individual components

Usage

This principle applies after uploading extensions to cloud storage, to verify the distribution pipeline works end-to-end. It is the fourth step in the Extension Development and Distribution workflow, following compression and upload, and preceding release promotion.

Typical scenarios:

• CI/CD pipelines run loading verification after uploading extensions to the nightly/staging bucket
• Release managers verify extensions before promoting from nightly to production
• Extension developers test their extensions against a custom repository during development

Theoretical Basis

• Integration testing of distribution pipelines -- testing the pipeline as a whole rather than individual components catches interface errors and configuration mismatches that unit tests miss.
• Smoke testing -- a minimal set of tests that verify basic functionality, providing rapid feedback on whether the artifact is fundamentally broken without running a comprehensive test suite.
• End-to-end validation -- exercising the same code path that end users will follow (SQL INSTALL and LOAD commands) ensures that the verified behavior matches the user experience.

Related Pages

• Implementation:Duckdb_Duckdb_Extension_Upload_Test