“These guys figured out how to do asynchronous training…” — The Changelog: Software Development, Open Source at 23:55
“These guys figured out how to do asynchronous training where you can have slower GPUs mixed with faster GPUs distributed across the world using something called a swarm training mechanism that can all somehow come together and train a model with 1% or less loss in accuracy, which is very, very good in terms of…”
0:00
2:02:37