Implementation:Duckdb Duckdb Extension Upload Single

Field	Value
source	`scripts/extension-upload-single.sh`, `scripts/extension-upload-all.sh`
domains	Extension_Development, Distribution
last_updated	2026-02-07

Overview

Concrete tool for compressing and uploading DuckDB extensions to S3 provided by shell scripts. This implementation handles gzip (or brotli for WASM) compression of signed extension binaries and their upload to versioned, platform-specific paths in Amazon S3 buckets.

Code Reference

Single Extension Upload

Source: scripts/extension-upload-single.sh (lines 70-101)

This script handles the compression and upload of a single signed extension binary:

Compresses the signed extension with gzip (or brotli for WASM targets)
Uploads the compressed file to the appropriate S3 path using the AWS CLI
Sets appropriate content-type and content-encoding headers

# Simplified logic from extension-upload-single.sh
# For native platforms:
gzip < "${SIGNED_EXTENSION}" > "${EXTENSION_NAME}.duckdb_extension.gz"

# Upload to versioned, platform-specific S3 path
aws s3 cp \
    "${EXTENSION_NAME}.duckdb_extension.gz" \
    "s3://duckdb-extensions/${DUCKDB_VERSION}/${DUCKDB_PLATFORM}/${EXTENSION_NAME}.duckdb_extension.gz"

# For WASM platform:
brotli < "${SIGNED_EXTENSION}" > "${EXTENSION_NAME}.duckdb_extension.br"

aws s3 cp \
    "${EXTENSION_NAME}.duckdb_extension.br" \
    "s3://duckdb-extensions/${DUCKDB_VERSION}/wasm/${EXTENSION_NAME}.duckdb_extension.br"

Batch Upload

Source: scripts/extension-upload-all.sh (lines 1-51)

This script iterates over all built extensions in the build directory and calls extension-upload-single.sh for each one, enabling batch upload of all extensions for a given platform in a single CI step.

# Simplified logic from extension-upload-all.sh
for ext_dir in build/release/extension/*/; do
    EXTENSION_NAME=$(basename "$ext_dir")
    EXTENSION_PATH="${ext_dir}/${EXTENSION_NAME}.duckdb_extension"
    if [ -f "$EXTENSION_PATH" ]; then
        scripts/extension-upload-single.sh \
            "$EXTENSION_PATH" "$DUCKDB_PLATFORM" "$DUCKDB_VERSION"
    fi
done

I/O Contract

API

Single extension upload:

scripts/extension-upload-single.sh <extension_path> <platform> <duckdb_version>

Batch upload:

scripts/extension-upload-all.sh <platform> <duckdb_version>

Environment Variables

Variable	Description	Required
`DUCKDB_EXTENSION_SIGNING_PK`	Base64-encoded RSA private key for signing (used in the signing step before upload)	Yes (for signed uploads)
`AWS_ACCESS_KEY_ID`	AWS access key for S3 upload	Yes
`AWS_SECRET_ACCESS_KEY`	AWS secret key for S3 upload	Yes

S3 URL Pattern

s3://duckdb-extensions/<duckdb_version>/<duckdb_platform>/<ext_name>.duckdb_extension.gz

Examples:

s3://duckdb-extensions/v0.10.0/linux_amd64/httpfs.duckdb_extension.gz
s3://duckdb-extensions/v0.10.0/osx_arm64/parquet.duckdb_extension.gz
s3://duckdb-extensions/v0.10.0/windows_amd64/json.duckdb_extension.gz
s3://duckdb-extensions/v0.10.0/wasm/httpfs.duckdb_extension.br

Inputs

Signed extension binary (.duckdb_extension file with metadata and signature)
S3 credentials (via environment variables)
DuckDB version string
Platform identifier string

Outputs

Gzipped (or Brotli-compressed for WASM) extension binary uploaded to the versioned S3 path
The uploaded file is publicly accessible at the corresponding HTTPS URL for DuckDB runtime download

Usage Examples

Uploading a single extension:

export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

scripts/extension-upload-single.sh \
    build/release/extension/httpfs/httpfs.duckdb_extension \
    linux_amd64 \
    v0.10.0

Uploading all extensions for a platform:

export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

scripts/extension-upload-all.sh linux_amd64 v0.10.0

Verifying the upload:

# Check that the extension is accessible
aws s3 ls s3://duckdb-extensions/v0.10.0/linux_amd64/httpfs.duckdb_extension.gz

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment