Skip to content
Past-Token Prediction for Long-Context Robot Policies episode artwork

May 20, 2025 15 min

Past-Token Prediction for Long-Context Robot Policies

from Best AI papers explained Hosted by Enoch H. Kang

This research presents Past Token Prediction (PTP), an auxiliary technique designed to improve long-context diffusion policies for robots learning tasks through imitation. The core idea is to explicitly train the policy to predict past actions along with future ones, which helps address the issue of modern diffusion policies failing to capture strong temporal dependencies. A multi-stage training strategy is introduced, separating visual encoder training from long-context policy training using cached embeddings to enhance efficiency. Additionally, PTP is used as a self-verification mechanism during inference by selecting candidate actions that best match previously executed actions. Experiments show this method significantly boosts performance and training speed on various simulated and real-world tasks, especially those requiring memory of past events.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Past-Token Prediction for Long-Context Robot Policies

0:00 15:59

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Best AI papers explained?

This episode is 15 minutes long.

When was this Best AI papers explained episode published?

This episode was published on May 20, 2025.

Can I download this Best AI papers explained episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.