DSP Lecture 21: Gradient descent and LMS

ECSE-4530 Digital Signal Processing Rich Radke, Rensselaer Polytechnic Institute Lecture 21: Gradient descent and LMS (11/13/14) 0:00:22 Recap: the Wiener filter 0:04:42 Estimating R and p in practice 0:06:45 The filter taps change over time based on data 0:08:40 How does the filter converge? 0:12:07 Steepest descent (gradient descent) 0:15:03 Basic equation 0:17:05 Step size considerations 0:19:30 Steepest descent for the Wiener filter 0:22:10 The final result 0:23:41 Comments on convergence 0:24:57 Convergence is related to the eigenvalues of R 0:27:01 Revisiting convergence problems 0:31:11 These kinds of optimization problems are common throughout engineering 0:36:05 The LMS (least-mean-square) algorithm 0:37:54 Estimating R from data 0:41:30 The LMS equations 0:44:16 Comments on convergence 0:45:49 Tap-input power 0:46:44 Adaptive step sizes Follows Section 13.2 of the textbook (Proakis and Manolakis, 4th ed.).