Implementation:Eventual Inc Daft Utils

Knowledge Sources	Eventual_Inc_Daft
Domains	Utilities, Data_Processing
Last Updated	2026-02-08 14:00 GMT

Overview

Concrete tool providing shared utility functions for column input normalization, Arrow version detection, Ray state detection, and numpy datetime conversion across the Daft codebase.

Description

The utils module contains general-purpose helper functions used throughout the Daft Python layer:

column_input_to_expression / column_inputs_to_expressions: Normalize string column names and Expression objects into a uniform list of Expressions, used by DataFrame operations that accept flexible column inputs.
get_arrow_version: Returns the installed PyArrow version as a tuple for feature detection.
detect_ray_state: Checks whether Ray is initialized or running inside a Ray job/worker.
np_datetime64_to_timestamp: Converts numpy datetime64 values to integer timestamps with appropriate time units.
in_notebook: Detects Jupyter notebook environments.

Usage

These utilities are primarily consumed internally by DataFrame methods, but `column_input_to_expression` is useful when building custom logic that needs to normalize user-provided column references.

Code Reference

Source Location

Repository: Eventual_Inc_Daft
File: daft/utils.py
Lines: 1-197

Signature

ColumnInputType = Expression | str
ManyColumnsInputType = ColumnInputType | Iterable[ColumnInputType]

def column_input_to_expression(column: ColumnInputType) -> Expression:
    """Converts a column-like object to a daft column expression."""

def column_inputs_to_expressions(columns: ManyColumnsInputType) -> list[Expression]:
    """Normalizes inputs to a list of Expressions."""

def get_arrow_version() -> tuple[int, ...]:
    """Returns the installed PyArrow version as a tuple."""

def detect_ray_state() -> tuple[bool, bool]:
    """Returns (ray_is_available, in_ray_worker)."""

def in_notebook() -> bool:
    """Check if running in a Jupyter notebook."""

def np_datetime64_to_timestamp(dt: np.datetime64) -> tuple[int, PyTimeUnit | None]:
    """Convert numpy datetime64 to value since unix epoch."""

Import

from daft.utils import column_input_to_expression, column_inputs_to_expressions, get_arrow_version

I/O Contract

Inputs

Name	Type	Required	Description
column	ColumnInputType (str or Expression)	Yes	A column name or Expression to normalize
columns	ManyColumnsInputType	Yes	One or more column inputs to normalize to a list

Outputs

Name	Type	Description
Expression	daft.Expression	Normalized column expression from string or pass-through Expression
list[Expression]	list	List of normalized column expressions

Usage Examples

Normalizing Column Inputs

from daft.utils import column_input_to_expression, column_inputs_to_expressions

# String input is converted to col("name")
expr = column_input_to_expression("name")

# Mixed inputs are normalized
exprs = column_inputs_to_expressions(["name", "age"])

Detecting Ray State

from daft.utils import detect_ray_state

ray_available, in_worker = detect_ray_state()
if ray_available and not in_worker:
    import daft
    daft.set_runner_ray()

Semantic Links

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment