Skip to content
Title: EP 61 | MIT 6.036: Gradient Descent & Optimization in Practice episode artwork

Oct 23, 2026 32 min

Title: EP 61 | MIT 6.036: Gradient Descent & Optimization in Practice

from AI Bites: The Academic Series Hosted by Jack Lakkapragada

Welcome to Week 5 of MIT 6.036! Now that we've framed machine learning as loss minimization, how do we actually find the parameters that sit at the bottom of the bowl? We explore the geometry of loss landscapes, analyze step-size dynamics, derive analytical gradients for Logistic Regression, and examine why Stochastic Gradient Descent (SGD) uses random noise to outpace traditional Batch GD. Key Topics: Intuition and update rules for 1D and High-Dimensional Gradient Descent. Convexity, Local Optima, and the critical role of learning rates ($\eta$). Computing analytical gradients for Logistic Regression ($\nabla_\theta J_{lr}$ and $\frac{\partial J}{\partial \theta_0}$). Stochastic Gradient Descent (SGD) convergence conditions and why noise prevents local traps. Disclaimer: Note: This is an AI-generated discussion created using Google's NotebookLM, Gemini, and other AI tools, based on the freely and publicly available MIT 6.036 - Introduction to Machine Learning course material and personal study notes.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Title: EP 61 | MIT 6.036: Gradient Descent & Optimization in Practice

0:00 32:39

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of AI Bites: The Academic Series?

This episode is 32 minutes long.

When was this AI Bites: The Academic Series episode published?

This episode was published on October 23, 2026.

Can I download this AI Bites: The Academic Series episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.