Gradient Descent in 3 minutes

Visual and intuitive overview of the Gradient Descent algorithm. This simple algorithm is the backbone of most machine learning applications. ------------------- References: - AI plays hide and seek: https://openai.com/blog/emergent-tool-use/ - AI plays Dota 2: https://openai.com/five/ - InterFaceGAN: https://www.youtube.com/watch?v=uoftpl3Bj6w - Boyd and Vandenberghe's book on Convex Optimization (Sections 9.2 and 9.3): https://web.stanford.edu/class/ee364a/lectures/unconstrained.pdf https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf ------------------ Timestamps: - 0:00 Intro - 0:18 Problem Formulation - 1:10 Gradient Descent - 2:12 Flavors of Gradient Descent ------------------- Music: www.bensound.com This video would not have been possible without the help of Gökçe Dayanıklı.