Title: EP 61 | MIT 6.036: Gradient Descent & Optimization in Practice
from AI Bites: The Academic Series Hosted by Jack Lakkapragada
Welcome to Week 5 of MIT 6.036! Now that we've framed machine learning as loss minimization, how do we actually find the parameters that sit at the bottom of the bowl? We explore the geometry of loss landscapes, analyze step-size dynamics, derive analytical gradients for Logistic Regression, and examine why Stochastic Gradient Descent (SGD) uses random noise to outpace traditional Batch GD. Key Topics: Intuition and update rules for 1D and High-Dimensional Gradient Descent. Convexity, Local Optima, and the critical role of learning rates ($\eta$). Computing analytical gradients for Logistic Regression ($\nabla_\theta J_{lr}$ and $\frac{\partial J}{\partial \theta_0}$). Stochastic Gradient Descent (SGD) convergence conditions and why noise prevents local traps. Disclaimer: Note: This is an AI-generated discussion created using Google's NotebookLM, Gemini, and other AI tools, based on the freely and publicly available MIT 6.036 - Introduction to Machine Learning course material and personal study notes.
Embed this episode
No transcript yet.