EPISODE · Jan 14, 2025 · 24 MIN
Training Large Language Models to Reason in Continuous Latent Space
from Deep Papers · host Arize AI
LLMs have typically been restricted to reason in the "language space," where chain-of-thought (CoT) is used to solve complex reasoning problems. But a new paper argues that language space may not always be the best for reasoning. In this paper read, we cover an exciting new technique from a team at Meta called Chain of Continuous Thought—also known as "Coconut." In the paper, "Training Large Language Models to Reason in a Continuous Latent Space" explores the potential of allowing LLMs to reason in an unrestricted latent space instead of being constrained by natural language tokens.Read a full breakdown of Coconut on our blog, or join us live for the next paper reading. Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.
Embed this episode
What this episode covers
LLMs have typically been restricted to reason in the "language space," where chain-of-thought (CoT) is used to solve complex reasoning problems. But a new paper argues that language space may not always be the best for reasoning. In this paper read, we cover an exciting new technique from a team at Meta called Chain of Continuous Thought—also known as "Coconut." In the paper, "Training Large Language Models to Reason in a Continuous Latent Space" explores the potential of allowing LLMs to rea...
NOW PLAYING
Training Large Language Models to Reason in Continuous Latent Space
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.