EPISODE · Apr 26, 2026 · 11 MIN
ZeRO: Removing Memory Redundancy from Data Parallel Training
from Mastering Language Models: From Architecture to Optimization
A deep dive into ZeRO’s memory model: optimizer states, gradients, and parameters are not sacred copies; they can be partitioned across workers.
Embed this episode
NOW PLAYING
ZeRO: Removing Memory Redundancy from Data Parallel Training
0:00
11:04
1×
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.
Frequently Asked Questions
How long is this episode of Mastering Language Models: From Architecture to Optimization?
This episode is 11 minutes long.
When was this Mastering Language Models: From Architecture to Optimization episode published?
This episode was published on April 26, 2026.
Can I download this Mastering Language Models: From Architecture to Optimization episode?
Yes. Use the download control on the episode player to save the publisher-provided media file.
URL copied to clipboard!