Implementation:TobikoData Sqlmesh DbtLoader Load

Knowledge Sources	SQLMesh SQLMesh Docs
Domains	Data_Engineering, Dbt_Migration
Last Updated	2026-02-07 00:00 GMT

Overview

Concrete tool for discovering and loading dbt project assets into SQLMesh model format provided by SQLMesh.

Description

The DbtLoader class extends SQLMesh's base Loader to implement dbt-specific project loading logic. Its load method orchestrates the discovery of all dbt assets (models, seeds, sources, macros, tests) and converts them into SQLMesh models using the DbtContext and model conversion methods. The loader handles multi-package dbt projects, applies caching strategies to avoid redundant conversions, and tracks file modification times to invalidate stale cached models.

The loader creates a bridge between dbt's project structure and SQLMesh's execution engine, ensuring that dbt models can be executed through SQLMesh's plan/apply workflow while maintaining dbt's semantics for materializations, incremental strategies, and macro expansion.

Usage

Use DbtLoader when you need to load a dbt project into a SQLMesh context for execution or analysis. This is typically invoked automatically when creating a SQLMesh context with a dbt project, but can also be used directly for custom loading workflows. The loader is designed to be used once per context initialization, with subsequent reloads detecting and reprocessing only changed files.

Code Reference

Source Location

• Repository: sqlmesh
• File: sqlmesh/dbt/loader.py

Signature

class DbtLoader(Loader):
    def __init__(
        self,
        context: GenericContext,
        path: Path,
        profiles_dir: t.Optional[Path] = None
    ) -> None:
        ...

    def load(self) -> LoadedProject:
        ...

Import

from pathlib import Path
from sqlmesh.dbt.loader import DbtLoader
from sqlmesh.core.context import Context

I/O Contract

Inputs

Name	Type	Required	Description
context	GenericContext	Yes	SQLMesh context providing configuration and execution environment
path	Path	Yes	Path to dbt project root directory containing dbt_project.yml
profiles_dir	Path	No	Override path to profiles.yml directory; defaults to ~/.dbt

Outputs

Name	Type	Description
loaded_project	LoadedProject	Container with converted SQLMesh models, macros, and project metadata

Usage Examples

Basic Usage with Context

from pathlib import Path
from sqlmesh.core.context import Context

# SQLMesh context automatically detects dbt project and uses DbtLoader
dbt_project_path = Path("/path/to/dbt/project")
context = Context(paths=dbt_project_path)

# Models are now loaded and accessible
for model_name, model in context.models.items():
    print(f"Model: {model_name}, Kind: {model.kind}")

Direct Loader Usage

from pathlib import Path
from sqlmesh.dbt.loader import DbtLoader
from sqlmesh.core.context import Context

# Create context
context = Context(paths=Path("/dbt/project"))

# Instantiate DbtLoader directly
loader = DbtLoader(
    context=context,
    path=Path("/dbt/project"),
    profiles_dir=Path.home() / ".dbt"
)

# Load project
loaded_project = loader.load()

# Access loaded models
print(f"Loaded {len(loaded_project.models)} models")
for model_fqn, model in loaded_project.models.items():
    print(f"  {model_fqn}: {model.kind.name}")

# Access loaded macros
print(f"Loaded {len(loaded_project.macros)} Python macros")
print(f"Loaded Jinja macros from {len(loaded_project.jinja_macros.packages)} packages")

Multi-Package Project Loading

from pathlib import Path
from sqlmesh.dbt.loader import DbtLoader
from sqlmesh.core.context import Context

# Load dbt project with multiple packages
context = Context(paths=Path("/dbt/project"))
loader = DbtLoader(context=context, path=Path("/dbt/project"))
loaded = loader.load()

# Models are organized by package internally
# but accessible as flat dictionary
for fqn, model in loaded.models.items():
    # FQN includes package prefix for non-root models
    print(f"Model: {fqn}")
    print(f"  Package: {model.project}")
    print(f"  Name: {model.name}")
    print(f"  Kind: {model.kind}")

Handling Load Errors

from pathlib import Path
from sqlmesh.dbt.loader import DbtLoader
from sqlmesh.core.context import Context
from sqlmesh.core.config import ConfigError

try:
    context = Context(paths=Path("/dbt/project"))
    loader = DbtLoader(context=context, path=Path("/dbt/project"))
    loaded = loader.load()

    print(f"Successfully loaded {len(loaded.models)} models")

except ConfigError as e:
    print(f"Configuration error: {e}")
    # Common issues:
    # - Invalid dbt_project.yml
    # - Missing profiles.yml
    # - Invalid model configurations

except Exception as e:
    print(f"Load error: {e}")
    # Could indicate:
    # - Syntax errors in model SQL
    # - Invalid macro references
    # - Circular dependencies

Accessing Converted Model Details

from pathlib import Path
from sqlmesh.core.context import Context

context = Context(paths=Path("/dbt/project"))

# Iterate through models
for fqn, model in context.models.items():
    print(f"\nModel: {fqn}")
    print(f"  Kind: {model.kind.name}")
    print(f"  Dialect: {model.dialect}")

    # Check if incremental
    if model.kind.is_incremental:
        print(f"  Time Column: {model.time_column}")
        print(f"  Partitioned By: {model.partitioned_by}")

    # Check dependencies
    print(f"  Dependencies: {', '.join(model.depends_on)}")

    # Check for audits (converted dbt tests)
    if model.audits:
        print(f"  Audits: {', '.join(model.audits)}")

Related Pages

Implements Principle

• Principle:TobikoData_Sqlmesh_Dbt_Model_Discovery_And_Mapping

• Environment:TobikoData_Sqlmesh_Dbt_Compatibility