https://www.patreon.com/ahmdelemam
Agenda
Batch & Stream Processing
Historical Overview
- Transactional Databases
- Data Warehouse & ETLs
Data pipelines
- What are data pipelines and How do they work?
- What is DAG?
- What are the Upstream and Downstream?
Batch Processing Example
- Map-Reduce
- Apache Airflow
- Apache Airflow The overall Architecture
- DAG Anatomy
Intro to Stream Processing