Skip to content
Thinking Machines Lab-On-Policy Distillation: Efficient Post-Training for LLMs episode artwork

Oct 28, 2025 6 min

Thinking Machines Lab-On-Policy Distillation: Efficient Post-Training for LLMs

from Steven AI Talk Hosted by Steven

The text from "On-Policy Distillation" introduces a method for training smaller "student" Language Models (LLMs) to achieve expert performance by combining the strengths of existing post-training techniques. The authors explain the traditional LLM training stages—pre-training, mid-training, and post-training—and compare two post-training approaches: on-policy training (like Reinforcement Learning, or RL, which offers sparse feedback) and off-policy training (like supervised fine-tuning/distillation, which can suffer from compounding errors). The core innovation, on-policy distillation, samples trajectories from the student model but uses a high-performing "teacher" model to grade every token, providing a dense reward signal that is significantly more compute-efficient than traditional RL. This technique is shown to be effective for training models in mathematical reasoning and for continual learning tasks, such as preserving instruction-following abilities while incorporating new domain knowledge.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Thinking Machines Lab-On-Policy Distillation: Efficient Post-Training for LLMs

0:00 6:48

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Steven AI Talk?

This episode is 6 minutes long.

When was this Steven AI Talk episode published?

This episode was published on October 28, 2025.

Can I download this Steven AI Talk episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.