Implementation:Eventual Inc Daft PyO3 Root Module

Knowledge Sources	Eventual_Inc_Daft PyO3 User Guide
Domains	FFI, Rust_Python_Bridge
Last Updated	2026-02-08 14:00 GMT

Overview

Concrete tool for bridging the Rust engine to Python by registering all sub-crate modules, configuring the jemalloc global allocator, initializing the logging and telemetry systems, and populating the function registry.

Description

The src/lib.rs file is the root PyO3 module that produces the `daft.daft` native extension. It:

Configures jemalloc as the global allocator on non-MSVC targets with tuned parameters for background thread management and decay timings.
Defines the `#[pymodule] fn daft()` entry point that initializes the Python-Rust bridge.
Calls `register_modules()` on approximately 30 sub-crates to expose their types and functions to Python.
Populates the global FUNCTION_REGISTRY with all built-in function families (numeric, float, URI, image, binary, list, UTF-8, JSON, serde, temporal, distance, similarity, tokenize, file, coalesce, monotonically_increasing_id).
Exposes utility pyfunction entries: `version()`, `build_type()`, `refresh_logger()`, `get_max_log_level()`, `set_compute_runtime_num_worker_threads()`.

Usage

This module is not called directly by users. It is automatically loaded when Python imports `daft.daft` (the compiled native extension). Developers modify this file when adding new Rust sub-crates or registering new function families.

Code Reference

Source Location

Repository: Eventual_Inc_Daft
File: src/lib.rs
Lines: 1-188

Signature

#[cfg(feature = "python")]
pub mod pylib {
    #[pymodule]
    fn daft(py: Python, m: &Bound<PyModule>) -> PyResult<()> {
        // Initializes logger, OpenTelemetry, registers all sub-crate modules,
        // populates the function registry
    }

    #[pyfunction]
    pub fn version() -> &'static str { ... }

    #[pyfunction]
    pub fn build_type() -> &'static str { ... }

    #[pyfunction]
    pub fn refresh_logger(py: Python) -> PyResult<()> { ... }

    #[pyfunction]
    pub fn get_max_log_level() -> &'static str { ... }

    #[pyfunction]
    pub fn set_compute_runtime_num_worker_threads(num_threads: usize) -> PyResult<()> { ... }
}

Import

# Loaded automatically by the daft package
from daft.daft import version, build_type, refresh_logger

I/O Contract

Inputs

Name	Type	Required	Description
py	Python GIL token	Yes	PyO3 Python interpreter reference (provided by the runtime)
m	&Bound<PyModule>	Yes	The root `daft` Python module being initialized

Outputs

Name	Type	Description
Initialized module	PyModule	The `daft.daft` module with all Rust types and functions registered
Function registry	Global state	Populated with all built-in expression functions

Usage Examples

Checking Version from Python

from daft.daft import version, build_type

print(f"Daft version: {version()}")
print(f"Build type: {build_type()}")

Registering a New Sub-Crate (Developer Guide)

// In src/lib.rs, inside the daft() function:
// 1. Add the register_modules call
daft_new_crate::register_modules(m)?;

// 2. If it has functions, register them:
functions_registry.register::<daft_new_crate::NewFunctions>();

Semantic Links

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment