Skip to content
Bo Zhao: Scalable and Flexible Distributed Reinforcement Learning Systems episode artwork

Oct 22, 2023 51 min

Bo Zhao: Scalable and Flexible Distributed Reinforcement Learning Systems

from Finnish Center for Artificial Intelligence FCAI Hosted by DASER

Abstract: Efficient machine learning (ML) systems translate data into value for decision making. Recent breakthroughs in large language models (GPT 4, Llama 2, ChatGPT) and remarkable outcomes of reinforcement learning (RL) in real-world settings (AlphaGo, AlphaFold, RLHF) have shown that scalable model training on large GPU/TPU clusters is critical to obtain state-of-the-art performance. This talk aims to answer the question “how to co-design multiple layers of the software/system stack to improve the scalability and performance of ML systems”. Specifically, it addresses the challenges to build (1) flexible distributed RL systems to accelerate and parallelize the RL training loop and (2) statement management libraries to transparently change the GPU device allocation and multi-dimensional parallelism (i.e., data/model/pipeline parallelism) without affecting the training result. Finally, the talk will discuss the open challenges to design and build large-scale RLHF systems. Speaker: Bo Zhao is a tenure-track Assistant Professor in the Department of Computer Science at Aalto University. His research focuses on efficient data-intensive systems at the intersection of scalable machine learning systems and distributed data management systems, as well as compilation-based optimization techniques. Bo’s long-term goal is to explore and understand the fundamental connections between data management and modern machine learning systems (e.g., ML training on quantum computers) to make decision-making more transparent, robust and efficient. Besides publishing at database and system conferences (e.g., SIGMOD and USENIX ATC), Bo’s research output has been applied into real-world settings (e.g.,smart grid management and public transportation) and integrated into industry software (e.g., Amazon Redshift cloud data warehouse and MindSpore ML framework).

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Bo Zhao: Scalable and Flexible Distributed Reinforcement Learning Systems

0:00 51:17

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Finnish Center for Artificial Intelligence FCAI?

This episode is 51 minutes long.

When was this Finnish Center for Artificial Intelligence FCAI episode published?

This episode was published on October 22, 2023.

Can I download this Finnish Center for Artificial Intelligence FCAI episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.