EPISODE · Oct 24, 2017 · 27 MIN
Synthetic Gradients Explained
from Siraj Raval
DeepMind released an optimization strategy that could become the most popular approach for training very deep neural networks, even more so than backpropagation. I don't think it got enough love, so i'm going to explain how it works myself and why i think it's so cool. Already know how backpropagation works? Skip to 14:10 Code for this video: https://github.com/llSourcell/synthetic_gradients_explained Please Subscribe! And like. And comment. Thats what keeps me going. Follow me on: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology/ Snapchat: @llSourcell More learning resources: https://iamtrask.github.io/2017/03/21/synthetic-gradients/ https://arxiv.org/abs/1703.00522 https://deepmind.com/blog/decoupled-neural-networks-using-synthetic-gradients/ Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Instagram: https://www.instagram.com/sirajraval/
What this episode covers
DeepMind released an optimization strategy that could become the most popular approach for training very deep neural networks, even more so than backpropagation. I don't think it got enough love, so i'm going to explain how it works myself and why i think it's so cool. Already know how backpropagation works? Skip to 14:10 Code for this video: https://github.com/llSourcell/synthetic_gradients_explained Please Subscribe! And like. And comment. Thats what keeps me going. Follow me on: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology/ Snapchat: @llSourcell More learning resources: https://iamtrask.github.io/2017/03/21/synthetic-gradients/ https://arxiv.org/abs/1703.00522 https://deepmind.com/blog/decoupled-neural-networks-using-synthetic-gradients/ Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Instagram: https://www.instagram.com/sirajraval/
NOW PLAYING
Synthetic Gradients Explained
No transcript for this episode yet