EPISODE · Jun 23, 2017 · 10 MIN
Second Order Optimization - The Math of Intelligence #2
from Siraj Raval
Gradient Descent and its variants are very useful, but there exists an entire other class of optimization techniques that aren't as widely understood. We'll learn about second order method variants, how they compare to first order methods, and implement our own in Python. Code for this video (with challenge): https://github.com/llSourcell/Second_Order_Optimization_Newtons_Method Alberto's Winning Code: https://github.com/alberduris Ivan's Runner up Code: https://github.com/PiaFraus Please Subscribe! And like. And comment. That's what keeps me going. Course Syllabus: https://github.com/llSourcell/The_Math_of_Intelligence More learning resources: https://web.stanford.edu/class/msande311/lecture13.pdf https://www.cs.toronto.edu/~hinton/csc2515/notes/lec6tutorial.pdf https://www.quora.com/In-mathematical-optimization-problems-the-first-derivative-is-often-used-Why-not-the-second-or-higher-order-derivatives https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization https://www.youtube.com/watch?v=28BMpgxn_Ec&t=444s https://www.youtube.com/watch?v=42zJ5xrdOqo&t=438s Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
What this episode covers
Gradient Descent and its variants are very useful, but there exists an entire other class of optimization techniques that aren't as widely understood. We'll learn about second order method variants, how they compare to first order methods, and implement our own in Python. Code for this video (with challenge): https://github.com/llSourcell/Second_Order_Optimization_Newtons_Method Alberto's Winning Code: https://github.com/alberduris Ivan's Runner up Code: https://github.com/PiaFraus Please Subscribe! And like. And comment. That's what keeps me going. Course Syllabus: https://github.com/llSourcell/The_Math_of_Intelligence More learning resources: https://web.stanford.edu/class/msande311/lecture13.pdf https://www.cs.toronto.edu/~hinton/csc2515/notes/lec6tutorial.pdf https://www.quora.com/In-mathematical-optimization-problems-the-first-derivative-is-often-used-Why-not-the-second-or-higher-order-derivatives https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization https://www.youtube.com/watch?v=28BMpgxn_Ec&t=444s https://www.youtube.com/watch?v=42zJ5xrdOqo&t=438s Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ And please support me on Patreon: https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
NOW PLAYING
Second Order Optimization - The Math of Intelligence #2
No transcript for this episode yet