CORRECTION: we are SUBTRACTING the learning_rate * gradient of last iteration(s) NOT adding.
We are still adding the "tweaks" because, by definition, the tweaks are already the negative of the learning_rate * gradient of last iteration(s).
#machinelearning #ml