EPISODE · Sep 4, 2025 · 43 MIN
Training a 1 trillion parameter model
from Pretrained · host Pierce Freeman & Richard Diehl Martinez
Kimi K2 and Moonshot AI's history, avoiding loss spikes during training, the muon optimizer, and data parallelism
Embed this episode
Ready to play
Training a 1 trillion parameter model
0:00
43:08
1×
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.
Frequently Asked Questions
How long is this episode of Pretrained?
This episode is 43 minutes long.
When was this Pretrained episode published?
This episode was published on September 4, 2025.
Can I download this Pretrained episode?
Yes. Use the download control on the episode player to save the publisher-provided media file.
URL copied to clipboard!