混合张量专家数据并行方法优化混合专家训练

from AI Podcast · host weedge

深入探讨 DeepSpeed-TED，一种新颖的三维混合并行框架，用于训练具有大型基础模型的混合专家模型。我们讨论了内存优化、通信优化以及与现有方法的性能比较。

Episode metadata supplied by the publisher feed · Published Jan 4, 2025

Embed this episode

Attribution link and audio player

Ready to play

混合张量专家数据并行方法优化混合专家训练

0:00 5:19

1×

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Similar Episodes

No similar episodes found.

No similar podcasts found.

This episode is 5 minutes long.

This episode was published on January 4, 2025.

Yes. Use the download control on the episode player to save the publisher-provided media file.

URL copied to clipboard!