Heuristic:Dagster io Dagster Lazy Import Pattern
| Knowledge Sources | |
|---|---|
| Domains | Performance, CLI_Startup |
| Last Updated | 2026-02-10 12:00 GMT |
Overview
Performance-critical pattern requiring function-scoped imports for heavy modules to minimize CLI startup time and avoid circular imports.
Description
Dagster enforces a mandatory lazy import pattern for specific modules that are expensive to load at import time. These modules must be imported inside functions rather than at the top of the file. This is a strict coding convention that directly impacts CLI startup time and user experience. The pattern also addresses circular import resolution and optional dependency handling.
Usage
Use this heuristic when writing or reviewing Dagster code that imports any of the modules on the mandatory lazy-import list. It is also applicable when adding new dependencies to Dagster that may be slow to import, or when resolving circular import errors.
The Insight (Rule of Thumb)
- Action: ALWAYS import the following modules inside functions, never at the top of the file:
jinja2requestsdagster_cloud_cli.*urllib.requestyamltyperpydantic(for Dagster core code)
- Value: Reduces CLI startup time significantly. Top-level import of these modules adds 100ms+ to every
dagsterCLI invocation. - Trade-off: Slightly more verbose code at call sites. Minor performance cost on first function call (one-time import overhead).
- Acceptable uses of function-scoped imports:
- Circular import resolution
- Optional dependency handling
- Expensive module loading
- CLI startup optimization
Reasoning
Dagster's CLI (dagster dev, dagster asset materialize, etc.) must start quickly for a good developer experience. Python's import system eagerly evaluates module-level code, so importing heavy modules at the top of frequently-imported files adds latency to every CLI invocation, even when those modules are not needed for the current command.
Empirical testing by the Dagster team showed that top-level imports of modules like jinja2, requests, and pydantic each add significant startup overhead. Since Dagster's core is imported by every CLI command, these costs compound.
The TYPE_CHECKING block pattern is used for type annotations that reference these modules:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import requests # Only imported during type checking, not at runtime
Code Evidence
From .claude/coding_conventions.md:
Performance-Critical Lazy Imports - MUST use function-scoped imports for:
- jinja2, requests, dagster_cloud_cli.*, urllib.request, yaml, typer, pydantic
Example of correct lazy import pattern:
def render_template(template_str: str, context: dict) -> str:
import jinja2 # Lazy import - do NOT move to top of file
template = jinja2.Template(template_str)
return template.render(context)