Hey, data engineers! Apache Spark is excellent for data transformations, especially when handling large datasets. However, setting up Spark can be challenging. That's why there are services like Azure Databricks that simplify the process. But what if you prefer not to use third-party solutions like Databricks and want to stick with Microsoft offerings? Is it possible? The answer is yes - we have Spark Pools in Azure Synapse Analytics!
Join me in the 39th episode of my free DP-203 course, where I discuss Azure Synapse Analytics Spark Pools and compare them to Azure Databricks.
▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬
My LinkedIn profile: https://www.linkedin.com/in/piotr-tybulewicz-81a8793/
GitHub with my drawings: https://github.com/TybulOnAzure/DP-203
Apache Spark in Azure Synapse Analytics: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-overview
▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬
Join this channel to get access to perks:
https://www.youtube.com/channel/UCLnXq-Fr-6rAsCitq9nYiGg/join
▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬
00:00 Introduction
00:53 Announcement
01:56 Spark revisited
06:37 Creating spark pool
15:47 Integration with ADLSg2
22:03 Notebooks
28:32 Synapse vs Databricks
51:10 Summary