Azure Data Factory Real Time Scenarios | Azure Data Factory | Azure Data Lake | Azure Data Engineer
In this video, I walk you through Azure Data Factory real-time scenarios using Azure Data Factory, Azure Data Lake, and APIs. We’ll cover key activities like Copy, ForEach, If Condition, GetMetadata, Filter, SetVariable, and more—showing you how to build complete end-to-end ETL pipelines in ADF.
Timestamps:
0:00 Introduction
05:56 Azure Free Account
08:23 Azure Tutorial
17:39 Scenario-1 Incremental Loading in ADF from Azure Data Lake
42:42 GetMetadata, ForEach, If Condition Activities in ADF
1:06:02 End To End Dynamic ETL Pipeline in Azure Data Factory
1:18:02 Scenario-2 Copy Only The Missing files in ADF
1:34:32 Scenario-3 Fetch files with Dates using Azure Data Factory
1:39:10 Scenario-4 Delete files with Dates using Azure Data Factory
1:56:58 Scenario-5 Storing the number of files in a Variable
2:02:02 Scenario-6 Dynamic Column Mapping in Azure Data Factory
WATCH MY PREVIOUS VIDEOS
➡️Azure End To End Data Project
https://youtu.be/uc-u_juRg-w?si=sO_CavjnhNaSnZri
➡️Databricks Tutorial
https://youtu.be/7pee6_Sq3VY?si=pPBrTnE2R8PhhHu5
➡️PySpark Full Course
https://youtu.be/94w6hPk7nkM?si=i3u_8ZWkGf6fiGyh
REPOSITORY
➡️GitHub
https://github.com/anshlambagit/ADF-RealTime-Scenarios
=======================================
Join this channel to SUPPORT MY HARDWORK:
https://www.youtube.com/channel/UCu7lQE-L5gzt8aD7zuuufjw/join
=======================================
Connect with ME
LinkedIn - https://www.linkedin.com/in/ansh-lamba-793681184/
Telegram - https://t.me/anshlambadatafam
=========================
For COLLABORATION 👇
anshlambacollab@gmail.com
=========================
⭐Hashtags⭐
#azure #azuredatafactory #dataengineering