Jump to content

Connect Leeroopedia MCP: Equip your AI agents to search best practices, build plans, verify code, diagnose failures, and look up hyperparameter defaults.

Implementation:Astronomer Astronomer cosmos RenderConfig Init

From Leeroopedia


Metadata

Field Value
Page Type Implementation
Repository astronomer-cosmos
Source File cosmos/config.py, Lines L50-157
Implements Principle:Astronomer_Astronomer_cosmos_Render_Configuration
Import from cosmos.config import RenderConfig

Overview

RenderConfig is the concrete tool provided by the astronomer-cosmos library for controlling how a dbt project graph is parsed, filtered, and rendered into Airflow tasks. It is a dataclass that encapsulates all rendering preferences, including node selection/exclusion, test behavior, load method, invocation mode, dataset emission, and source rendering behavior.

This configuration object is consumed by cosmos's graph rendering engine to transform a dbt project's dependency graph into an Airflow-compatible task structure.

Source Location

Property Value
Repository astronomer-cosmos
File cosmos/config.py
Lines L50-157
Class RenderConfig (decorated with @dataclass)

Code Reference

Signature

@dataclass
class RenderConfig:
    emit_datasets: bool = True
    test_behavior: TestBehavior = TestBehavior.AFTER_EACH
    load_method: LoadMode = LoadMode.AUTOMATIC
    invocation_mode: InvocationMode = InvocationMode.DBT_RUNNER
    select: list[str] = field(default_factory=list)
    exclude: list[str] = field(default_factory=list)
    selector: str | None = None
    dbt_deps: bool | None = None
    node_converters: dict[DbtResourceType, Callable[..., Any]] | None = None
    node_conversion_by_task_group: bool | None = True
    dbt_executable_path: str | Path = get_system_dbt()
    env_vars: dict[str, str] | None = None
    dbt_project_path: InitVar[str | Path | None] = None
    dbt_ls_path: Path | None = None
    enable_mock_profile: bool = True
    source_rendering_behavior: SourceRenderingBehavior = SourceRenderingBehavior.NONE
    source_pruning: bool = False
    should_detach_multiple_parents_tests: bool = False
    enable_owner_inheritance: bool | None = True

Import Statements

from cosmos.config import RenderConfig
from cosmos.constants import TestBehavior, LoadMode, InvocationMode
from cosmos.constants import SourceRenderingBehavior
from cosmos.dbt.graph import DbtResourceType

Parameters

Parameter Type Default Description
emit_datasets bool True Whether to emit Airflow datasets for cross-DAG dependency management.
test_behavior TestBehavior AFTER_EACH How dbt tests are organized relative to models (AFTER_EACH, AFTER_ALL, NONE, BUILD).
load_method LoadMode AUTOMATIC Method for discovering the dbt project graph (AUTOMATIC, DBT_LS, MANIFEST, CUSTOM, DBT_LS_FILE).
invocation_mode InvocationMode DBT_RUNNER How dbt is invoked during rendering (DBT_RUNNER, SUBPROCESS).
select list[str] [] List of dbt node selection strings to include.
exclude list[str] [] List of dbt node selection strings to exclude.
selector None None Named selector from selectors.yml.
dbt_deps None None Whether to run dbt deps before rendering. None defers to ProjectConfig.install_dbt_deps.
node_converters None None Custom callables for converting specific dbt resource types into Airflow operators.
node_conversion_by_task_group None True Whether to group converted nodes into Airflow TaskGroups.
dbt_executable_path Path get_system_dbt() Path to the dbt executable used during rendering.
env_vars None None Environment variables for the dbt rendering process.
dbt_project_path InitVar None Init-only variable for the dbt project path (used during initialization, not stored).
dbt_ls_path None None Path to a pre-computed dbt ls output file.
enable_mock_profile bool True Whether to use a mock profile during rendering to avoid needing real credentials.
source_rendering_behavior SourceRenderingBehavior NONE How source nodes are rendered in the DAG.
source_pruning bool False Whether to prune unnecessary source nodes from the rendered graph.
should_detach_multiple_parents_tests bool False Whether to detach tests with multiple parent models into standalone tasks.
enable_owner_inheritance None True Whether child nodes inherit owner metadata from parent nodes.

I/O Contract

Inputs

All parameters are optional with sensible defaults. The primary inputs that shape rendering behavior are:

  • Filtering: select, exclude, and selector control which nodes appear in the rendered DAG.
  • Topology: test_behavior controls the structure of test tasks relative to model tasks.
  • Parsing: load_method determines how the dbt project graph is discovered.
  • Dataset Integration: emit_datasets and source_rendering_behavior control Airflow dataset integration.

Outputs

A RenderConfig instance that is consumed by cosmos's graph rendering engine. The instance provides:

  • All rendering preferences as typed attributes.
  • Validation logic ensuring consistent configuration (e.g., selector cannot be used simultaneously with select/exclude).

Usage Examples

Basic Selection and Exclusion

from cosmos.config import RenderConfig

# Render only staging models, excluding a specific model
render_config = RenderConfig(
    select=["path:models/staging"],
    exclude=["path:models/staging/stg_legacy_orders"],
)

Test Behavior Configuration

from cosmos.config import RenderConfig
from cosmos.constants import TestBehavior

# Run all tests after all models complete (reduces task count)
render_config = RenderConfig(
    test_behavior=TestBehavior.AFTER_ALL,
)

# Skip tests entirely in the orchestration DAG
render_config_no_tests = RenderConfig(
    test_behavior=TestBehavior.NONE,
)

Load Method Configuration

from cosmos.config import RenderConfig
from cosmos.constants import LoadMode

# Use manifest for fast DAG parsing (requires manifest_path in ProjectConfig)
render_config = RenderConfig(
    load_method=LoadMode.MANIFEST,
)

# Use dbt ls for accurate, real-time graph discovery
render_config_ls = RenderConfig(
    load_method=LoadMode.DBT_LS,
)

Tag-Based Selection with Graph Operators

from cosmos.config import RenderConfig
from cosmos.constants import TestBehavior

# Select all nodes tagged 'daily' plus their upstream dependencies
render_config = RenderConfig(
    select=["+tag:daily"],
    test_behavior=TestBehavior.AFTER_EACH,
    emit_datasets=True,
)

Complete DAG Integration

from cosmos import DbtTaskGroup, ProjectConfig, ProfileConfig, RenderConfig
from cosmos.constants import TestBehavior, LoadMode

render_config = RenderConfig(
    select=["path:models/marts"],
    exclude=["tag:wip"],
    test_behavior=TestBehavior.AFTER_EACH,
    load_method=LoadMode.MANIFEST,
    emit_datasets=True,
    source_rendering_behavior=SourceRenderingBehavior.NONE,
)

dbt_tasks = DbtTaskGroup(
    project_config=ProjectConfig(
        dbt_project_path="/usr/local/airflow/dags/dbt/my_project",
        manifest_path="/usr/local/airflow/dags/dbt/target/manifest.json",
    ),
    profile_config=profile_config,
    render_config=render_config,
)

Related Pages

Implements Principle

Complementary Implementations

Requires Environment

Uses Heuristic

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment