Skip to content
Neural Neural Scaling Laws episode artwork

Mar 31, 2026 19 min

Neural Neural Scaling Laws

from Mechanical Dreams Hosted by Mechanical Dirk

In this episode: • Introduction to Downstream Scaling Laws: Linda and Professor Norris introduce the paper and discuss the limitations of traditional parametric scaling laws for predicting downstream task performance. • The Token-Level Secret: Linda explains how NeuNeu uses token-level probabilities instead of average validation loss to capture critical distributional signals. • Architecture Deep Dive: The hosts break down the model components, detailing the CNN loss encoder and the Transformer time-series extrapolator using compute gaps. • Results and Zero-Shot Generalization: Norris is won over by the 38 percent error reduction and NeuNeu's impressive ability to generalize to unseen models like the Pythia family. • Ranking Models and Future Outlook: The episode concludes with a discussion on quantile regression, practical model ranking, and the dream of foundation models for training dynamics.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Neural Neural Scaling Laws

0:00 19:15

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Mechanical Dreams?

This episode is 19 minutes long.

When was this Mechanical Dreams episode published?

This episode was published on March 31, 2026.

Can I download this Mechanical Dreams episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.