EPISODE · May 27, 2017 · 8 MIN
Which Activation Function Should I Use?
from Siraj Raval
All neural networks use activation functions, but the reasons behind using them are never clear! Let's discuss what activation functions are, when they should be used, and what the difference between them is. Sample code from this video: https://github.com/llSourcell/Which-Activation-Function-Should-I-Use Please subscribe! And like. And comment. That's what keeps me going. More Learning resources: http://www.kdnuggets.com/2016/08/role-activation-function-neural-network.html http://cs231n.github.io/neural-networks-1/ https://www.quora.com/What-is-the-role-of-the-activation-function-in-a-neural-network https://stats.stackexchange.com/questions/115258/comprehensive-list-of-activation-functions-in-neural-networks-with-pros-cons https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions https://stackoverflow.com/questions/9782071/why-must-a-nonlinear-activation-function-be-used-in-a-backpropagation-neural-net https://papers.nips.cc/paper/874-how-to-choose-an-activation-function.pdf http://neuralnetworksanddeeplearning.com/chap4.html https://medium.com/towards-data-science/activation-functions-in-neural-networks-58115cda9c96 https://medium.com/autonomous-agents/mathematical-foundation-for-activation-functions-in-artificial-neural-networks-a51c9dd7c089 Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
What this episode covers
All neural networks use activation functions, but the reasons behind using them are never clear! Let's discuss what activation functions are, when they should be used, and what the difference between them is. Sample code from this video: https://github.com/llSourcell/Which-Activation-Function-Should-I-Use Please subscribe! And like. And comment. That's what keeps me going. More Learning resources: http://www.kdnuggets.com/2016/08/role-activation-function-neural-network.html http://cs231n.github.io/neural-networks-1/ https://www.quora.com/What-is-the-role-of-the-activation-function-in-a-neural-network https://stats.stackexchange.com/questions/115258/comprehensive-list-of-activation-functions-in-neural-networks-with-pros-cons https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions https://stackoverflow.com/questions/9782071/why-must-a-nonlinear-activation-function-be-used-in-a-backpropagation-neural-net https://papers.nips.cc/paper/874-how-to-choose-an-activation-function.pdf http://neuralnetworksanddeeplearning.com/chap4.html https://medium.com/towards-data-science/activation-functions-in-neural-networks-58115cda9c96 https://medium.com/autonomous-agents/mathematical-foundation-for-activation-functions-in-artificial-neural-networks-a51c9dd7c089 Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
NOW PLAYING
Which Activation Function Should I Use?
No transcript for this episode yet