Implementation:Duckdb Duckdb Extension Promote Nightly

Field	Value
source	`scripts/extension-upload-from-nightly.sh`
domains	Extension_Development, Release_Management
last_updated	2026-02-07

Overview

Concrete tool for promoting tested DuckDB extensions from nightly to production S3 buckets. This implementation uses a shell script that copies extension artifacts from the nightly (staging) S3 bucket to the production S3 bucket, iterating over all supported platforms to ensure complete promotion for a given DuckDB version.

Code Reference

Source: scripts/extension-upload-from-nightly.sh (lines 1-109)

This script performs the promotion by recursively copying extensions from the nightly bucket to the production bucket for each supported platform:

# Simplified logic from extension-upload-from-nightly.sh
DUCKDB_VERSION="$1"

# List of all supported platforms
PLATFORMS=(
    "linux_amd64"
    "linux_amd64_gcc4"
    "linux_arm64"
    "osx_amd64"
    "osx_arm64"
    "windows_amd64"
    "wasm_eh"
    "wasm_mvp"
)

FROM_BUCKET="s3://duckdb-extensions-nightly"
TO_BUCKET="s3://duckdb-extensions"

for platform in "${PLATFORMS[@]}"; do
    echo "Promoting extensions for platform: ${platform}"

    aws s3 cp --recursive \
        "${FROM_BUCKET}/${DUCKDB_VERSION}/${platform}/" \
        "${TO_BUCKET}/${DUCKDB_VERSION}/${platform}/"
done

The script handles several additional concerns:

Error handling -- if the copy fails for any platform, the script reports the failure and continues with remaining platforms, then exits with a non-zero status
Platform enumeration -- the list of platforms is maintained in the script and must be updated when new platforms are added
Version parameter -- the DuckDB version is passed as a command-line argument, enabling promotion of any version

I/O Contract

API

Promotion command:

scripts/extension-upload-from-nightly.sh <duckdb_version>

Underlying S3 operation:

aws s3 cp --recursive \
    s3://duckdb-extensions-nightly/<version>/<platform>/ \
    s3://duckdb-extensions/<version>/<platform>/

Parameters

Parameter	Description	Example
`DUCKDB_VERSION`	Version of DuckDB to promote extensions for	`v0.10.0`
FROM (implicit)	Nightly S3 bucket prefix	`s3://duckdb-extensions-nightly/`
TO (implicit)	Production S3 bucket prefix	`s3://duckdb-extensions/`

External Dependencies

Tool	Purpose	Version
`aws` CLI	S3 copy operations	v2.x
S3 bucket access	Read access to nightly bucket, write access to production bucket	N/A

Inputs

Tested nightly extensions on S3 (at s3://duckdb-extensions-nightly/<version>/<platform>/)
Target production bucket (at s3://duckdb-extensions/<version>/<platform>/)
AWS credentials with appropriate S3 permissions

Outputs

Extensions available at production S3 URLs, ready for end-user INSTALL commands
Each promoted file is byte-for-byte identical to its nightly counterpart

Usage Examples

Promoting a release version:

# Promote all extensions for v0.10.0 from nightly to production
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

scripts/extension-upload-from-nightly.sh v0.10.0

Verifying promotion for a specific platform:

# List promoted extensions for linux_amd64
aws s3 ls s3://duckdb-extensions/v0.10.0/linux_amd64/

Promoting and then verifying end-user access:

# After promotion, verify that end users can install the extension
./build/release/duckdb -c "
    INSTALL 'httpfs';
    LOAD 'httpfs';
    SELECT 'httpfs loaded from production' AS status;
"

Manual promotion for a single platform (for debugging):

# Copy only linux_amd64 extensions
aws s3 cp --recursive \
    s3://duckdb-extensions-nightly/v0.10.0/linux_amd64/ \
    s3://duckdb-extensions/v0.10.0/linux_amd64/

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment