MENU

Fun & Interesting

Execute Dbt with Airflow and Cloud Run

Video Not Working? Fix It Now

*How to Run dbt on Cloud Run with Apache Airflow and Cloud Composer* Learn how to seamlessly execute dbt pipelines using *Apache Airflow* and *Cloud Composer* in this hands-on guide. We demonstrate two powerful approaches: 1. *CloudRunJobOperator Approach*: Execute your entire dbt pipeline as a single task with the CloudRunJobOperator, then transition smoothly to subsequent tasks. 2. *Cosmos Library Approach*: Break down your dbt pipeline into Airflow tasks with a DbtTaskGroup, mirroring all dbt models as individual tasks for detailed monitoring and control. Discover best practices, optimize your workflows, and simplify dbt orchestration in the cloud. Perfect for data engineers and cloud enthusiasts working with *Google Cloud Platform (GCP)*. The link the GitHub repository: https://github.com/tosun-si/world-cup-qatar-elt-airflow-dbt #googlecloud #airflow #dbt #cloudrun

Comment