10+ hours of FREE Fabric Training: https://www.skool.com/microsoft-fabric/classroom/d154aad4?md=3b108b0e216c46c88d891407ccd8647b
Learn Apache Spark in Microsoft Fabric in the 30 days of September.
Here's the playlist for this series if you want to catchup: https://www.youtube.com/playlist?list=PLug2zSFKZmV1z1zUCs0iUEHgtmHJIwFiQ
Link to the GitHub for this series: https://github.com/LearnMicrosoftFabric/YouTube/tree/main/learn_spark_in_fabric
Kaggle Dataset: https://www.kaggle.com/datasets/dineshpiyasamara/sentiment-analysis-dataset
pySpark Machine Learning documentation:
https://spark.apache.org/docs/latest/ml-guide.html
Spark is the engine behind both the Data Engineering AND the Data Science experiences in Microsoft Fabric, so in September I'll be walking you through Apache Spark: what it is, why you should learn it, how to use it, and how it integrates into Microsoft Fabric.
No previous Spark knowledge is required, some basic Python would be useful!
#pyspark #microsoftfabric #apachespark
Here's the schedule:
1 Welcome - https://youtu.be/v5pJLJg9j6c
2 Why Spark? https://youtu.be/fFN7uvra8tg
3 Components of Spark - https://youtu.be/p3Ss_P6AyG0
4 Spark DataFrame - https://youtu.be/51EMJfnx47Q
5 Read Files into DataFrame - https://youtu.be/ja5MFgpdTAw
6 Read/Write to Lakehouse Table - https://youtu.be/bou77s-9b7c
7 Basic DataFrame Operations - https://youtu.be/tvM5MYgmips
8 DataFrame Filtering - https://youtu.be/rvgcJ81KYFY
9 GroupBy and Aggregate Functions - https://youtu.be/fuxlLQmuccY
10 Handling missing values - https://youtu.be/H83lW_RkeRU
11 Joining DataFrames - https://youtu.be/4bP5wqRH3AU
12 Time-series data - https://youtu.be/PlIUuuCtKm4
13 Spark SQL - https://youtu.be/eoj86nVraeI
14-16 Spark Machine Learning - https://youtu.be/f3-Xr3tPPsc
20 Configuring Spark - https://youtu.be/HIW8MvX4pWw
21 Autotuning Spark Configuration - https://youtu.be/PGbC_Z7Fhv0
22 Library Management - https://youtu.be/9pfBPXSRwlM
23 High-concurrency mode - https://youtu.be/4Fi1hzLNya4
24 Spark Scala - https://youtu.be/SuzmZszsZhc
25 Fabric MSSparkUtils - https://youtu.be/50z--o0R5dM
26 Monitoring Spark - https://youtu.be/CslPrMXxpUY
30 FINALE QnA - https://youtu.be/KmL5qhAGhAs
Timeline
0:00 Coming up...
0:22 Intro to SparkML
2:08 Reviewing the documentation
4:28 Introducing the Kaggle dataset
5:44 Read data & Train/Test Split
7:47 Basic Concepts of SparkML
9:55 Feature Engineering in SparkML
15:55 SparkML Pipelines
16:30 Logistic Regression Model
17:26 Model testing and evaluation
19:45 Hyperparameter tuning and cross-validation
20.50 Future work: SynapseML
21:15 Future work: Saving models in Fabric
21:48 Wrapup
--BROWSE MY OTHER FABRIC PLAYLISTS--
DATA ENGINEERING https://www.youtube.com/playlist?list=PLug2zSFKZmV1NvKfnRzG9e3Fl-8QLD5MK
END-TO-END FABRIC PROJECT https://www.youtube.com/playlist?list=PLug2zSFKZmV1BHk129X_2xi53ZRwQaTkb
INTRO TO MICROSOFT FABRIC https://www.youtube.com/playlist?list=PLug2zSFKZmV0Yaya7NxRQfrrPtfF2vj0K
DATA FACTORY https://www.youtube.com/playlist?list=PLug2zSFKZmV3FkUFDxlyrfSJMQ3CeqyEF
--LINKEDIN--
Not following the LinkedIn page yet? Here's the link: https://www.linkedin.com/company/learnmicrosoftfabric/
--ABOUT WILL--
Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric.
--SUBSCRIBE--
Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.
https://youtube.com/@LearnMicrosoftFabric?sub_confirmation=1