Implementation:Duckdb Duckdb Brotli Platform

Knowledge Sources	Duckdb_Duckdb Brotli
Domains	Compression, Third_Party
Last Updated	2026-02-07 12:00 GMT

Overview

The Brotli Platform header provides compiler-specific and platform-specific macros, type definitions, and utility functions that abstract away differences across operating systems, CPU architectures, and compilers for the entire Brotli compression library.

Description

brotli_platform.h is the central portability layer for the Brotli compression library as bundled within DuckDB. It is a 543-line header-only file that defines a comprehensive set of preprocessor macros and inline utility functions to handle:

• Compiler detection and feature gating: Macros such as BROTLI_GNUC_HAS_BUILTIN, BROTLI_MSVC_VERSION_CHECK, BROTLI_INTEL_VERSION_CHECK, and others are used to conditionally enable compiler-specific optimizations. These are derived from the Hedley portability header project (with permission from Evan Nemerson).

• Branch prediction hints: BROTLI_PREDICT_TRUE(x) and BROTLI_PREDICT_FALSE(x) wrap __builtin_expect on supported compilers (GCC, Clang, Intel, ARM, IBM, TI, TinyCC) to guide the CPU branch predictor on hot and cold code paths.

• Inlining control: BROTLI_INLINE maps to __attribute__((__always_inline__)), __forceinline, or equivalent pragma depending on the compiler. BROTLI_NOINLINE provides the inverse, and BROTLI_MAYBE_INLINE offers a portable inline keyword.

• Architecture and endianness detection: The header detects x86, x64, ARMv7, ARMv8, PowerPC64, RISC-V 64, LoongArch64, and NEON availability. It determines endianness via __BYTE_ORDER__, platform-specific includes, or build-time overrides (BROTLI_BUILD_BIG_ENDIAN, BROTLI_BUILD_LITTLE_ENDIAN, BROTLI_BUILD_ENDIAN_NEUTRAL).

• Register-width type: brotli_reg_t is defined as uint64_t on 64-bit targets or uint32_t on 32-bit targets, providing a native-width integer for performance-critical loops.

• Unaligned memory access: Portable functions BrotliUnalignedRead16, BrotliUnalignedRead32, BrotliUnalignedRead64, and BrotliUnalignedWrite64 use memcpy to safely perform unaligned reads and writes. The macro BROTLI_UNALIGNED_READ_FAST indicates whether the target platform handles unaligned access efficiently (true for x86, x64, ARMv7, ARMv8, RISC-V 64, LoongArch64).

• Endian-aware loads: BROTLI_UNALIGNED_LOAD16LE, BROTLI_UNALIGNED_LOAD32LE, BROTLI_UNALIGNED_LOAD64LE, and BROTLI_UNALIGNED_STORE64LE provide little-endian memory access, using byte-swaps on big-endian platforms.

• Build options: Compile-time flags control optimizations: BROTLI_BUILD_32_BIT, BROTLI_BUILD_64_BIT, BROTLI_BUILD_NO_RBIT, BROTLI_BUILD_NO_UNALIGNED_READ_FAST, BROTLI_DEBUG, and BROTLI_ENABLE_LOG.

• Debugging support: When BROTLI_DEBUG or BROTLI_ENABLE_LOG is defined, the header includes assert.h and stdio.h and defines BROTLI_DCHECK and BROTLI_LOG macros for internal diagnostics.

Usage

DuckDB includes this header as part of its vendored Brotli third-party dependency. Every Brotli source file -- encoder, decoder, dictionary, and transform modules -- includes brotli_platform.h either directly or transitively. It ensures that the Brotli library compiles correctly and performs optimally across all platforms that DuckDB supports (Linux, macOS, Windows, various CPU architectures). DuckDB's Parquet file reader and writer use Brotli compression, and this platform header is the foundation that enables Brotli to operate portably within DuckDB's build system.

Code Reference

Source Location

• Repository: Duckdb_Duckdb
• File: third_party/brotli/common/brotli_platform.h (543 lines)

Signature

// Branch prediction macros
#define BROTLI_PREDICT_TRUE(x)  (__builtin_expect(!!(x), 1))
#define BROTLI_PREDICT_FALSE(x) (__builtin_expect(x, 0))

// Restrict qualifier
#define BROTLI_RESTRICT __restrict

// Inlining control
#define BROTLI_INLINE     BROTLI_MAYBE_INLINE __attribute__((__always_inline__))
#define BROTLI_NOINLINE   __attribute__((__noinline__))

// Native register-width type
#define brotli_reg_t uint64_t  // on 64-bit targets
#define brotli_reg_t uint32_t  // on 32-bit targets

// Portable unaligned memory access
static BROTLI_INLINE uint16_t BrotliUnalignedRead16(const void* p);
static BROTLI_INLINE uint32_t BrotliUnalignedRead32(const void* p);
static BROTLI_INLINE uint64_t BrotliUnalignedRead64(const void* p);
static BROTLI_INLINE void BrotliUnalignedWrite64(void* p, uint64_t v);

// Little-endian load macros
#define BROTLI_UNALIGNED_LOAD16LE(p) ...
#define BROTLI_UNALIGNED_LOAD32LE(p) ...
#define BROTLI_UNALIGNED_LOAD64LE(p) ...
#define BROTLI_UNALIGNED_STORE64LE(p, v) ...

Import

#include "../common/brotli_platform.h"

I/O Contract

Inputs

Name	Type	Required	Description
p	const void*	Yes	Pointer to memory location for unaligned read/write functions
v	uint64_t	Yes (write)	Value to write for BrotliUnalignedWrite64
n	brotli_reg_t	Yes	Bit count for BitMask (used in bit_reader.h)

Outputs

Name	Type	Description
(return)	uint16_t / uint32_t / uint64_t	Value read from memory in platform-native or little-endian byte order
Macros	Preprocessor definitions	Architecture flags, endianness flags, optimization hints used at compile time

Usage Examples

// Typical usage within Brotli internals (e.g., bit_reader.h or decode.cpp)
#include "../common/brotli_platform.h"

// Using branch prediction hints in a hot loop
if (BROTLI_PREDICT_TRUE(available_in > 0)) {
    // fast path: process input
    brotli_reg_t bits = BROTLI_UNALIGNED_LOAD64LE(next_in);
    // ...
} else {
    // slow path: need more input
}

// Portable unaligned 32-bit read
uint32_t value = BrotliUnalignedRead32(ptr);

// Using native register type for loop counter
brotli_reg_t i;
for (i = 0; i < count; ++i) {
    // ...
}

Related Pages

• Principle:Duckdb_Duckdb_Brotli_Compression