Implementation:Astronomer Astronomer cosmos Gcp Cloud Run Job Operators

Knowledge Sources	Astronomer_Astronomer_cosmos
Domains	GCP, Cloud_Run, Operators
Last Updated	2026-02-07 17:00 GMT

Overview

Provides Airflow operators for executing dbt commands as Google Cloud Run jobs by combining the Cosmos abstract dbt interface with Airflow's native CloudRunExecuteJobOperator.

Description

The Gcp_Cloud_Run_Job_Operators module defines DbtGcpCloudRunJobBaseOperator, which inherits from both AbstractDbtBase and CloudRunExecuteJobOperator. This dual inheritance pattern allows the operator to assemble dbt CLI commands through the Cosmos abstract layer and execute them as Cloud Run jobs in Google Cloud Platform. The base operator translates dbt profile configuration, environment variables, and the constructed command into Cloud Run job overrides, passing them to the GCP API for execution.

Cloud Run jobs are a serverless execution model on GCP that provisions compute resources on demand, runs the container to completion, and then deallocates. This makes them well-suited for batch workloads like dbt transformations where you pay only for the compute time consumed.

Nine concrete operator classes extend the base, covering dbt sub-commands: build, ls, seed, snapshot, source freshness, run, test, run-operation, and clone. Each concrete class sets its own ui_color and ui_fgcolor for the Airflow graph view.

Usage

Use the GCP Cloud Run Job operators when your dbt workloads should run on Google Cloud Platform in a fully serverless manner. This is appropriate when you do not want to manage Kubernetes clusters (GKE) or dedicated compute instances and prefer the simplicity of Cloud Run's on-demand container execution. Typical scenarios include teams already invested in the GCP ecosystem who want minimal infrastructure overhead for their dbt orchestration.

Code Reference

Source Location

• Repository: Astronomer_Astronomer_cosmos
• File: cosmos/operators/gcp_cloud_run_job.py

Signature

class DbtGcpCloudRunJobBaseOperator(AbstractDbtBase, CloudRunExecuteJobOperator):
    """
    Base class for running dbt commands as GCP Cloud Run jobs.
    """

    def __init__(
        self,
        project_id: str,
        region: str,
        job_name: str,
        profile_config: ProfileConfig | None = None,
        command: list[str] | None = None,
        environment_variables: dict[str, str] | None = None,
        **kwargs,
    ) -> None:
        ...

Concrete operators:

class DbtBuildGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtLSGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtSeedGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtSnapshotGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtSourceGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtRunGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtTestGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtRunOperationGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...
class DbtCloneGcpCloudRunJobOperator(DbtGcpCloudRunJobBaseOperator): ...

Import

from cosmos.operators.gcp_cloud_run_job import DbtRunGcpCloudRunJobOperator

I/O Contract

Inputs

Name	Type	Required	Description
project_id	str	Yes	The GCP project ID where the Cloud Run job is defined.
region	str	Yes	The GCP region where the Cloud Run job will execute (e.g., "us-central1").
job_name	str	Yes	The name of the Cloud Run job resource to execute.
profile_config	ProfileConfig or None	No	Cosmos ProfileConfig object defining the dbt profile and target to use.
command	list[str] or None	No	Additional command fragments to pass to the dbt CLI invocation.
environment_variables	dict[str, str] or None	No	Extra environment variables injected into the Cloud Run job container at runtime.

Outputs

Name	Type	Description
execution_name	str	The name of the Cloud Run job execution that was created, available via XCom.
log_output	str	Standard output and error from the dbt command, captured through Cloud Logging.

Usage Examples

from cosmos.operators.gcp_cloud_run_job import DbtRunGcpCloudRunJobOperator

run_dbt = DbtRunGcpCloudRunJobOperator(
    task_id="dbt_run_on_cloud_run",
    project_id="my-gcp-project",
    region="us-central1",
    job_name="dbt-runner-job",
    environment_variables={
        "DBT_TARGET": "prod",
    },
)

from cosmos.operators.gcp_cloud_run_job import DbtBuildGcpCloudRunJobOperator

build_dbt = DbtBuildGcpCloudRunJobOperator(
    task_id="dbt_build_on_cloud_run",
    project_id="my-gcp-project",
    region="europe-west1",
    job_name="dbt-build-job",
)

Related Pages

• Environment:Astronomer_Astronomer_cosmos_Python_Airflow_Runtime