Skip to content
ShiQ: Bringing back Bellman to LLMs episode artwork

May 22, 2025 18 min

ShiQ: Bringing back Bellman to LLMs

from Best AI papers explained Hosted by Enoch H. Kang

This paper introduces ShiQ, a novel offline reinforcement learning algorithm designed for fine-tuning large language models (LLMs) by adapting traditional Q-learning methods. The authors address the challenges of applying Q-learning to LLMs, such as computational cost and initialization issues, by deriving theoretically grounded loss functions from Bellman equations. ShiQ enables off-policy, token-wise learning and is evaluated on various benchmarks, including multi-turn settings, where it demonstrates effectiveness compared to existing methods like DPO and CoPG. The paper details the theoretical basis of ShiQ and includes empirical results from both synthetic and real-world datasets.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

ShiQ: Bringing back Bellman to LLMs

0:00 18:03

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Best AI papers explained?

This episode is 18 minutes long.

When was this Best AI papers explained episode published?

This episode was published on May 22, 2025.

Can I download this Best AI papers explained episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.