Implementation:Eventual Inc Daft LazyImport
| Knowledge Sources | |
|---|---|
| Domains | Import_Management, Performance |
| Last Updated | 2026-02-08 14:00 GMT |
Overview
Concrete tool for deferring module imports until first attribute access, reducing startup time by avoiding eager loading of heavy optional dependencies.
Description
The LazyImport class provides a proxy object that defers `importlib.import_module` calls until the first attribute access via `__getattr__`. This avoids importing heavy optional dependencies like Ray, Unity Catalog, or other large packages at module load time. It supports pickling (via `__getstate__`/`__setstate__`), submodule auto-discovery, and a `module_available()` check that does not raise on missing packages.
Usage
Use `LazyImport` when declaring optional or heavy dependencies that should only be loaded when actually accessed. This is particularly useful for conditional imports like Ray (only needed with the Ray runner) or database connectors.
Code Reference
Source Location
- Repository: Eventual_Inc_Daft
- File: daft/lazy_import.py
- Lines: 1-78
Signature
class LazyImport:
"""Manages Optional Dependency imports."""
def __init__(self, module_name: str):
"""Create a lazy module proxy.
Args:
module_name: Fully qualified module name (e.g., "ray.data").
"""
def module_available(self) -> bool:
"""Check if the module can be imported without raising."""
def __getattr__(self, name: str) -> Any:
"""Load the module on first attribute access and delegate."""
def __getstate__(self) -> dict[str, Any]:
"""Pickle support: serialize only module name."""
def __setstate__(self, state: dict[str, Any]) -> None:
"""Pickle support: restore from module name."""
Import
from daft.lazy_import import LazyImport
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| module_name | str | Yes | Fully qualified Python module name to lazily import |
Outputs
| Name | Type | Description |
|---|---|---|
| LazyImport proxy | LazyImport | Object that behaves like the target module on attribute access |
| ImportError | Exception | Raised on first access if the module is not installed |
Usage Examples
Lazy Loading Ray
from daft.lazy_import import LazyImport
ray = LazyImport("ray")
# Check availability without importing
if ray.module_available():
# This triggers the actual import
ray.init(num_cpus=4)
else:
print("Ray is not installed")
Submodule Access
from daft.lazy_import import LazyImport
ray = LazyImport("ray")
# Accessing ray.data automatically creates a LazyImport("ray.data")
dataset = ray.data.from_items([1, 2, 3])