Azure End-To-End Data Engineering Project | Azure Data Factory | Databricks | PySpark
In this video, I walk you through a complete Azure End-To-End Data Engineering project from scratch, using latest technologies such as Azure Data Factory, Databricks, PySpark, Azure Data Lake, Apache Spark, and some Azure Data Engineer Interview Questions.
Timestamps:
0:00 Introduction
01:19 Azure Data Architecture
09:30 Data Understanding
13:52 Free Azure Account
16:22 Azure Data Fundamentals
29:56 Azure Data Factory Tutorial for Beginners
42:27 ETL Pipelines with Azure Data Factory
52:25 Azure Data Factory Real Time Scenarios
1:22:45 Azure Databricks Tutorial
1:36:38 Databricks Unity Catalog
1:51:09 Databricks Spark Cluster
1:55:10 Incremental Loading in Databricks with Autoloader
2:14:21 Spark Streaming
2:22:47 Data Ingestion using PySpark
2:33:05 Parameters using Databricks Utilities
2:47:01 Data Orchestration with Databricks Workflows
3:04:54 Pyspark Tutorial
3:17:21 Data Transformation using PySpark
3:36:19 Big Data Analytics with Apache Spark
3:54:22 Databricks Delta Live Tables
4:15:02 End to End Pipeline in Databricks
WATCH MY PREVIOUS VIDEOS
➡️Azure End To End Data Project
https://youtu.be/6_hXeNg9TJ0?si=9naCovTmgcZn0NQQ
➡️Databricks Tutorial
https://youtu.be/P5pEeR3xQpI?si=XbQYrYkrVp2jKjfE
➡️PySpark Full Course
https://youtu.be/94w6hPk7nkM?si=Y2EUnifXrZRK3MOj
➡️PySpark Interview Ques (2025)
https://youtu.be/fOCiis31Ng4?si=SNSxDEG1m_n8vH1p
REPOSITORY
➡️Original Dataset
https://public.tableau.com/app/learn/sample-data
➡️GitHub
https://github.com/anshlambagit/Netflix_Azure_Data_Engineering_Project/tree/main/RawData_AND_Notebooks
=======================================
Join this channel to SUPPORT MY HARDWORK:
https://www.youtube.com/channel/UCu7lQE-L5gzt8aD7zuuufjw/join
=======================================
Connect with ME
LinkedIn - https://www.linkedin.com/in/ansh-lamba-793681184/
Telegram - https://t.me/anshlambadatafam
=========================
For COLLABORATION 👇
[email protected]
=========================
⭐Hashtags⭐
#azure #databricks #azuredatafactory