Principle:Dagster io Dagster Dbt Test To Asset Check
| Attribute | Value |
|---|---|
| Title | Dbt Test To Asset Check |
| Category | Principle |
| Domains | Data_Engineering, dbt, Data_Quality |
| Repository | Dagster_io_Dagster |
Overview
Mechanism for automatically converting dbt test declarations into Dagster asset checks to unify data quality validation across SQL and Python assets.
Description
dbt's built-in test framework (not_null, unique, accepted_values, relationships) is automatically mapped to Dagster asset checks when using the dagster-dbt integration. Each dbt test becomes a Dagster AssetCheckSpec linked to the corresponding Dagster asset. Tests execute during dbt build and results are reported as Dagster check results. This unifies data quality observability in the Dagster UI regardless of whether checks are Python-based or SQL-based.
The mapping operates through the following process:
- Manifest parsing: The
build_dbt_specs()function reads the dbt manifest and identifies all test nodes, distinguishing between model tests and source tests. - Spec generation: For each dbt test, the
DagsterDbtTranslator.get_asset_check_spec()method creates anAssetCheckSpeclinked to the asset that the test validates. - Execution integration: When
dbt buildruns, test results are captured from dbt's event stream and converted to DagsterAssetCheckResultobjects. - Source test handling: Source tests can optionally be treated as asset checks (via
enable_source_tests_as_checks) or emitted as observations.
Usage
Use when dbt models already have test declarations (in schema YAML files) and you want those tests visible and enforceable in the Dagster UI alongside Python-based asset checks.
Key behaviors:
- Enabled by default:
DagsterDbtTranslatorSettings.enable_asset_checks = Truemeans dbt tests are automatically converted to Dagster asset checks. - Source tests are opt-in: By default, source tests emit observations, not check results. Set
enable_source_tests_as_checks = Trueto change this. - No additional configuration: Any dbt test declared in
schema.ymlautomatically appears in the Dagster UI.
Theoretical Basis
This implements a quality gateway pattern where tests defined in one system (dbt) are transparently surfaced in another (Dagster). The build_dbt_specs() function parses the dbt manifest to extract test metadata, then the DagsterDbtTranslator converts each test into an AssetCheckSpec.
Design principles:
- Zero-configuration default: Tests work without any additional configuration, following the principle of least surprise. dbt users expect their tests to run; Dagster users expect to see check results.
- Opt-in complexity: The
enable_source_tests_as_checksflag andget_asset_check_spec()override provide escape hatches for advanced use cases. - Unified observability: By mapping dbt tests to Dagster's check framework, all data quality signals appear in a single UI, regardless of their implementation language (SQL vs. Python).
- Consistent execution model: Tests run as part of
dbt build, maintaining dbt's execution semantics while exposing results through Dagster's event system.