Implementation:Duckdb Duckdb Generate C Api

Overview

Concrete tool for generating DuckDB C API headers from JSON function definitions. The script scripts/generate_c_api.py reads structured JSON files describing every public C API function, validates them, and produces four output header files that form DuckDB's public C interface.

Code Reference

Field	Value
Source	`scripts/generate_c_api.py` (lines 1--1037)
Language	Python 3
API	`python3 scripts/generate_c_api.py`
External Dependencies	`python3`, `json` (stdlib), `re` (stdlib), `glob` (stdlib), `argparse` (stdlib), `packaging` (for `Version`)

Key Constants

Constant	Value	Purpose
`DUCKDB_HEADER_OUT_FILE`	`src/include/duckdb.h`	Main unified C API header output
`DUCKDB_HEADER_C_OUT_FILE`	`src/include/duckdb_extension.h`	Extension C API header output
`DUCKDB_HEADER_GO_OUT_FILE`	`src/include/duckdb_go_extension.h`	Go extension header output
`DUCKDB_HEADER_EXT_INTERNAL_OUT_FILE`	`src/include/duckdb/main/capi/extension_api.hpp`	Internal extension API header output
`CAPI_FUNCTION_DEFINITION_FILES`	`src/include/duckdb/main/capi/header_generation/functions/*/.json`	Glob pattern for function definition JSON files (~40 files)
`EXT_API_DEFINITION_PATTERN`	`src/include/duckdb/main/capi/header_generation/apis/v1//.json`	Extension API version definitions
`BASE_HEADER_TEMPLATE`	`src/include/duckdb/main/capi/header_generation/header_base.hpp.template`	Template header that forms the base for generation
`ORIGINAL_FUNCTION_GROUP_ORDER`	List of 37 group names	Controls ordering of function groups in output for stable diffs

I/O Contract

Input: Function Definition JSON

Each JSON file under the functions/ directory defines a group of C API functions:

{
    "group": "open_connect",
    "deprecated": false,
    "description": "Functions to operate on databases, connections, ...",
    "entries": [
        {
            "name": "duckdb_open",
            "return_type": "duckdb_state",
            "params": [
                { "type": "const char *", "name": "path" },
                { "type": "duckdb_database *", "name": "out_database" }
            ],
            "comment": {
                "description": "Creates a new database ...\n",
                "param_comments": {
                    "path": "Path to the database file on disk.",
                    "out_database": "The result database object."
                },
                "return_value": "DuckDBSuccess on success or DuckDBError on failure."
            }
        }
    ]
}

JSON Field	Type	Description
`group`	string	Logical grouping name (must appear in `ORIGINAL_FUNCTION_GROUP_ORDER`)
`deprecated`	boolean	Whether the entire group is deprecated
`entries[].name`	string	C function name (must be globally unique)
`entries[].return_type`	string	C return type
`entries[].params[]`	array	List of `{"type": ..., "name": ...}` parameter objects
`entries[].comment`	object	Doxygen comment with `description`, `param_comments`, `return_value`

Input: Extension API Version JSON

Located under apis/v1/*/, each file defines functions available at a specific extension API version:

{
    "version": "v0.10.0",
    "entries": ["duckdb_open", "duckdb_close", "..."]
}

Output Files

Output File	Description
`src/include/duckdb.h`	Unified C API header with all function declarations, Doxygen comments, and type definitions
`src/include/duckdb_extension.h`	Extension-facing header with function pointers dispatched through `duckdb_ext_api_v1` struct
`src/include/duckdb_go_extension.h`	Go-specific extension header variant
`src/include/duckdb/main/capi/extension_api.hpp`	Internal C++ header defining the extension API dispatch struct

Key Functions

Function	Purpose
`parse_capi_function_definitions()`	Reads all JSON files, validates for duplicates, returns ordered function groups and a function map
`parse_ext_api_definitions()`	Reads extension API version JSON files, sorts by version
`parse_exclusion_list()`	Reads the exclusion list for functions omitted from extension API
`create_function_comment()`	Generates Doxygen-style comment block from JSON comment fields
`create_function_declaration()`	Generates C function declaration from JSON entry
`fetch_header_template_main()`	Loads the base header template and extracts content after the start marker

Usage Examples

Generate all C API headers from the repository root:

python3 scripts/generate_c_api.py

This reads all functions/*.json files, validates the definitions, and writes the four output headers. If a duplicate symbol is found or a new group is not registered in ORIGINAL_FUNCTION_GROUP_ORDER, the script exits with an error.

Typical Workflow

Add a new function entry to the appropriate JSON file under src/include/duckdb/main/capi/header_generation/functions/
If creating a new function group, add the group name to ORIGINAL_FUNCTION_GROUP_ORDER in the script
Run python3 scripts/generate_c_api.py
Verify the generated headers (duckdb.h, duckdb_extension.h, etc.)

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment