In this video, we will understand in detail what is Momentum Optimizer in Deep Learning.
Momentum Optimizer in Deep Learning is a technique that reduces the time taken to train a model.
The path of learning in mini-batch gradient descent is zig-zag, and not straight. Thus, some time gets wasted in moving in a zig-zag direction. Momentum Optimizer in Deep Learning smooth out the zig-zag path and make it much more straighter, thus reducing the time taken to train the model.
Momentum Optimizer uses Exponentially Weighted Moving Average, which averages out the vertical movement and the net movement is mostly in the horizontal direction. Thus zig-zag path becomes straighter.
In this video, we will also understand what Exponentially Weighted Moving Average is, and thus this video is a full in-depth explanation of Momentum Optimizer in Deep Learning.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
▶ Mini Batch Gradient Descent: https://youtu.be/kmb5FuFCZKE
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
▶ Watch Next Video on RMSprop Optimizer: https://youtu.be/ajI_HTyaCu8
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
✔ Improving Neural Network Playlist: https://www.youtube.com/watch?v=SOI39DEHGSk&list=PLuhqtP7jdD8DKUBtucBD0mGS7y0rT9alz&t=0s
✔ Complete Neural Network Playlist: https://www.youtube.com/watch?v=vtx1iwmOx10&t=284s
✔ Complete Logistic Regression Playlist: https://www.youtube.com/watch?v=U1omz0B9FTw&list=PLuhqtP7jdD8Chy7QIo5U0zzKP8-emLdny&t=0s
✔ Complete Linear Regression Playlist: https://www.youtube.com/watch?v=mlk0r...
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Timestamp:
0:00 Agenda
1:00 Why do we need Momentum?
2:53 Exponentially Weighted Moving Average
8:29 Momentum in Mini Batch Gradient Descent
9:50 Why Momentum works?
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Subscribe to my channel, because I upload a new Machine Learning video every week: https://www.youtube.com/channel/UCJFA...