Databricks Full Course (With UNITY CATALOG) | Azure Databricks Tutorial | Spark Streaming using PySpark
🔍 What You'll Learn:
This 4-hour tutorial on Databricks, focusing on the most in-demand topics such as Unity Catalog, Unity Metastore, Databricks Storage Credential, Databricks External Location, Volumes in Databricks, Cloning in Databricks, Loading Incremental Data using Autoloader, Delta Lake (in detail), along with real time scenarios and Databricks interview questions.
This video is perfect for both beginners and seasoned professionals looking to deepen their understanding of Databricks and its powerful features.
Data Source Link : https://github.com/anshlambagit/Databricks-Unity-Catalog/tree/main/Raw%20Data
Databricks Tutorial : https://youtu.be/7pee6_Sq3VY?si=R9RU3n_FXQFxbSem
PySpark Full Course : https://youtu.be/94w6hPk7nkM?si=Y2EUnifXrZRK3MOj
Azure Data Factory Full Course : https://youtu.be/8zIVOdKyoDA?si=XMgP7hLda_neTHWw
End-To-End Azure Data Engineering Project : https://youtu.be/0GTZ-12hYtU?si=_9Gyh6KvDt5HX_OT
Timestamps:
0:00 Introduction
06:07 Databricks Architecture (Control Plane and Data Plane)
11:13 Databricks Integration with Cloud
14:54 Azure Portal Free Account
18:22 Azure Portal Tutorial
24:29 Azure Data Lake Tutorial
27:58 Azure Databricks Free Account
31:43 Azure Databricks Overview
38:44 Databricks Spark Cluster
40:36 Unity Catalog Databricks
45:45 Databricks Unity Catalog vs Hive Metastore
56:04 Databricks Admin Console
59:09 Unity Catalog Metastore
1:07:09 Managed Table and External Table
1:11:36 Databricks Access Connector
1:13:31 RBAC Azure
1:21:58 Databricks Unity Cluster
1:23:12 Unity Catalog Databricks Tutorial
1:26:45 Databricks Notebook
1:27:43 Magic Commands in Databricks
1:31:56 Databricks Storage Credential
1:33:19 Databricks External Location
1:38:53 Databricks Real Time Scenarios
1:40:28 Scenario 1 (Managed Catalog, Managed Schema, Managed Table)
1:46:00 Scenario 2 (External Catalog, Managed Schema, Managed Table)
1:51:21 Scenario 3 (External Catalog, External Schema, Managed Table)
1:54:54 Scenario 4 (External Catalog, External Schema, External Table)
1:58:13 DROP and UNDROP in Databricks
2:01:10 Databricks Views
2:05:26 Read Data from ADLS using Databricks
2:14:22 Databricks Volumes Unity Catalog
2:21:13 Managed Volumes and External Volumes in Databricks
2:28:59 Delta Table in Databricks
2:40:53 Delta Lake Tutorial
2:41:53 Delta Log Databricks
2:44:51 CRUD operations in Delta Lake
2:47:10 Delta Lake Architecture
2:49:44 Data Versioning in Delta Lake
2:52:07 Time Travel in Delta Lake
2:54:00 Deletion Vectors Databricks
3:02:09 Delta Table Optimization using OPTIMIZE
3:08:37 Deep Clone and Shallow Clone Databricks
3:18:04 Incremental Data Ingestion with Autoloader
3:22:50 Spark Streaming Dataframe
3:24:13 Databricks Workflows and Jobs
Connect with ME - https://www.linkedin.com/in/ansh-lamba-793681184/
Please Hit the SUBSCRIBE button❤️to support me and my hard work.
⭐Hashtags⭐
#azure #databricks #dataengineering #dataengineer #data #apachespark #deltalake #azuredatabricks #azuredataengineer