EPISODE · Jun 3, 2017 · 9 MIN
The Evolution of Gradient Descent
from Siraj Raval
Which optimizer should we use to train our neural network? Tensorflow gives us lots of options, and there are way too many acronyms. We'll go over how the most popular ones work and in the process see how gradient descent has evolved over the years. Code from this video (with coding challenge): https://github.com/llSourcell/The_evolution_of_gradient_descent/ Please subscribe! And like. And comment. Thats what keeps me going. More learning resources: http://sebastianruder.com/optimizing-gradient-descent/ https://www.tensorflow.org/api_docs/python/tf/train/GradientDescentOptimizer http://machinelearningmastery.com/gradient-descent-for-machine-learning/ http://cs231n.github.io/optimization-1/ https://www.cs.toronto.edu/~hinton/csc2515/notes/lec6tutorial.pdf https://www.youtube.com/watch?v=umAeJ7LMCfU Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
What this episode covers
Which optimizer should we use to train our neural network? Tensorflow gives us lots of options, and there are way too many acronyms. We'll go over how the most popular ones work and in the process see how gradient descent has evolved over the years. Code from this video (with coding challenge): https://github.com/llSourcell/The_evolution_of_gradient_descent/ Please subscribe! And like. And comment. Thats what keeps me going. More learning resources: http://sebastianruder.com/optimizing-gradient-descent/ https://www.tensorflow.org/api_docs/python/tf/train/GradientDescentOptimizer http://machinelearningmastery.com/gradient-descent-for-machine-learning/ http://cs231n.github.io/optimization-1/ https://www.cs.toronto.edu/~hinton/csc2515/notes/lec6tutorial.pdf https://www.youtube.com/watch?v=umAeJ7LMCfU Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
NOW PLAYING
The Evolution of Gradient Descent
No transcript for this episode yet