The NEW way to ingest data with Airflow in a single task!
Get started with Airbyte with the dedicated Udemy course 👇 (FREE until June 3th)
https://www.udemy.com/course/the-complete-hands-on-introduction-to-airbyte/?couponCode=YOUTUBE_1
Or use the coupon YOUTUBE_1
Data ingestion is tricky, but it doesn't have to be!
Stop wasting time finding the right operator to transfer data.
Rule them all in a single task with Airflow and PyAirbyte!
In this video, you will:
✅ Transfer data from S3 to BigQuery efficiently with PyAirbyte
✅ Create Python Virtual Environments to save running time
✅ Use the ExternalPythonOperator to avoid dependency conflicts
✅ Check data in BigQuery with the BigQueryHook
And more!
No need to juggle between different operators or use custom scripts anymore!
🤖 The code:
https://robust-dinosaur-2ef.notion.site/The-new-way-to-ingest-data-with-Airflow-PUBLIC-bfe8f1359e4444279fac1a6b048c4edd?pvs=4
🏆 BECOME A PRO: https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/?couponCode=BIGPROMO1
👍 Smash the like button to become an Airflow Super Hero!
❤️ Subscribe to my channel to become a master of Airflow