End to End Azure Data Engineering Project by Sumit Sir

Link to Ultimate Data Engineering Program by Sumit Sir - https://trendytech.in/ultimate-big-data-masters-program-cloud-focused/ End to End Azure Data Engineering Project by Sumit Sir This includes both part1 and part2 (one shot) Github Link to the Datasets and reference code - https://github.com/bigdatabysumitm/Trendytech-Azure-Project Azure Project Timestamps 00:00:00 - Introduction 00:03:23 - Problem Statement and Domain Overview 00:29:48 - Role of a Data Engineer in the Project 00:32:02 - Overview of Datasets 00:56:28 - Solution Architecture Breakdown 01:16:00 - Technologies Used in the Project 01:18:04 - step 1 - Azure Storage Account 01:26:27 - Pipeline Implementation Details 02:35:11 - Summary of the Full Load Process 02:37:03 - Pipeline Design for Incremental Load 02:50:31 - Challenges with the Current Pipeline 02:58:34 - Introduction to Part 2 3:07:30 - Quick start of Part 2 3:08:55 - Recap 3:15:21 - Overview of the Bronze Layer 3:17:39 - Explanation of ICD and CPT Codes 3:20:20 - Transition from Bronze to Silver Layer 3:22:14 - Transition from Silver to Gold Layer 3:23:42 - Discussion on Enhancements 3:30:50 - Setup Overview 3:38:15 - Key Vault Setup and Usage 3:51:10 - Linked Services 3:57:52 - Datasets Overview 3:59:05 - Active and Inactive Flags 4:06:00 - Adding Other Datasets to Bronze Layer 4:29:54 - Bronze to Silver Layer 4:45:08 - Introduction to SCD Type 2 4:46:56 - Explanation of CDM (Common Data Model) 4:52:28 - Quality Checks 4:55:08 - Implementation of SCD Type 2 5:13:33 - Silver to Gold Layer 5:18:00 - Gold Layer Queries 5:21:20 - ADF (Azure Data Factory) Pipeline 5:36:04 - Syncing Code to GitHub Technology - Azure Data Engineering Stack Domain - Healthcare Revenue Cycle Management (RCM) RCM is the process that hospitals use to manage the financial aspects, from the time the patient schedules an appointment till the time the provider gets paid.. In This Project will be heavily using Azure Databricks & Azure Data Factory. We will be creating a metadata driven pipeline. Links to understand the KPI's & understand more on Accounts Receivable https://mdmanagementgroup.com/healthcare-accounts-receivable-management/ https://gentem.com/blog/revenue-cycle-kpis-definitions-and-benchmarks/ difference between ICD codes CPT codes https://www.simplepractice.com/blog/icd-codes-and-cpt-codes/ DO not forget to like the video & share with your friends!