Implementation:Dagster io Dagster DbtProjectComponent API
| Attribute | Value |
|---|---|
| Title | DbtProjectComponent API |
| Category | Implementation |
| Domains | Data_Engineering, dbt |
| Repository | Dagster_io_Dagster |
| Source | python_modules/libraries/dagster-dbt/dagster_dbt/components/dbt_project/component.py:L109
|
Overview
Concrete component for integrating dbt projects into Dagster's asset graph provided by the dagster-dbt library.
Description
The DbtProjectComponent is a StateBackedComponent and Resolvable dataclass that exposes a dbt project to Dagster as a set of assets. It is primarily configured via YAML in a defs.yaml file and handles the full lifecycle of dbt integration: project discovery, manifest parsing, asset/check spec generation, and dbt CLI execution.
Key responsibilities:
- Project resolution: Accepts a project path (string) or structured
DbtProjectArgsand resolves it to aDbtProjectorDbtProjectManager. - Spec generation: Calls
build_dbt_specs()to parse the dbt manifest and produceAssetSpecandAssetCheckSpecobjects. - Multi-asset creation: Wraps the dbt execution in a
@multi_assetdecorator withcan_subset=Truefor selective materialization. - CLI argument resolution: Resolves Jinja templates in CLI args at execution time, injecting partition keys and time windows.
Usage
The component is typically used via YAML configuration rather than direct Python import:
# defs.yaml
type: dagster_dbt.DbtProjectComponent
attributes:
project: ../analytics
translation:
key: "target/main/{{ node.name }}"
group_name: dbt
cli_args:
- "build"
prepare_if_dev: true
Scaffold a new integration via the CLI:
dg scaffold defs dagster_dbt.DbtProjectComponent --project-path <path>
Code Reference
Source Location
python_modules/libraries/dagster-dbt/dagster_dbt/components/dbt_project/component.py:L109
Signature
@public
@scaffold_with(DbtProjectComponentScaffolder)
@dataclass
class DbtProjectComponent(StateBackedComponent, dg.Resolvable):
"""Expose a DBT project to Dagster as a set of assets."""
project: Union[DbtProject, DbtProjectManager]
cli_args: list[Union[str, dict[str, Any]]] = field(default_factory=lambda: ["build"])
include_metadata: list[DbtMetadataAddons] = field(default_factory=list)
op: Optional[OpSpec] = None
translation: Optional[DbtProjectTranslation] = None
select: str = DBT_DEFAULT_SELECT
exclude: str = DBT_DEFAULT_EXCLUDE
selector: str = DBT_DEFAULT_SELECTOR
Import
from dagster_dbt import DbtProjectComponent
Typically used via YAML configuration rather than direct Python import.
I/O Contract
Inputs
- project: Path to dbt project directory containing
dbt_project.yml, models, and sources. Can be a string path or a structuredDbtProjectArgsmapping. - cli_args: List of arguments passed to the dbt CLI. Defaults to
["build"]. Supports Jinja templates for partition variables. - translation: Optional translation configuration for asset key mapping, group names, and descriptions.
- select / exclude / selector: dbt selection syntax for filtering which models to include.
- include_metadata: Optional list of metadata addons (e.g.,
"row_count","column_metadata").
Outputs
- AssetSpec objects: One per dbt model, seed, or snapshot, with proper dependency links to upstream Dagster assets.
- AssetCheckSpec objects: One per dbt test, linked to the corresponding asset.
- Materialization events: Streamed from dbt CLI execution during asset materialization.
Usage Examples
Basic YAML Configuration
type: dagster_dbt.DbtProjectComponent
attributes:
project: ../analytics
translation:
key: "target/main/{{ node.name }}"
group_name: dbt
cli_args:
- "build"
With Partition Variables
type: dagster_dbt.DbtProjectComponent
attributes:
project: ../analytics
cli_args:
- "build"
- "--full_refresh"
- "--vars":
start_date: "{{ partition_range_start }}"
end_date: "{{ partition_range_end }}"
With Metadata Addons
type: dagster_dbt.DbtProjectComponent
attributes:
project: ../analytics
cli_args:
- "build"
include_metadata:
- row_count
- column_metadata