Skip to content
According to Google DeepMind Can Language Models Perform Multi-Hop Reasoning Without Shortcuts? episode artwork

Dec 18, 2024 5 min

According to Google DeepMind Can Language Models Perform Multi-Hop Reasoning Without Shortcuts?

from New Paradigm: AI Research Summaries Hosted by James Bentley

This episode analyzes the research paper titled "Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?" by Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, and Mor Geva, affiliated with Google DeepMind, UCL, Google Research, and Tel Aviv University. The discussion examines whether large language models (LLMs) are capable of genuine multi-hop reasoning—connecting multiple pieces of information—without relying on shortcuts from their training data. To investigate this, the researchers developed the SOCRATES dataset, designed to evaluate the models' reasoning abilities in a shortcut-free environment. The findings reveal that while LLMs achieve high performance in tasks involving structured data, such as recalling countries, their effectiveness drops significantly with less structured data like years. Additionally, the study highlights a notable gap between latent multi-hop reasoning and explicit Chain-of-Thought reasoning, indicating that models may internally process information differently than how they articulate their reasoning. These insights underscore the current strengths and limitations of LLMs in complex reasoning tasks and suggest directions for future advancements in artificial intelligence research. This podcast is created with the assistance of AI, the producers and editors take every effort to ensure each episode is of the highest quality and accuracy. For more information on content and research relating to this episode please see: https://arxiv.org/pdf/2411.16679

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

According to Google DeepMind Can Language Models Perform Multi-Hop Reasoning Without Shortcuts?

0:00 5:48

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of New Paradigm: AI Research Summaries?

This episode is 5 minutes long.

When was this New Paradigm: AI Research Summaries episode published?

This episode was published on December 18, 2024.

Can I download this New Paradigm: AI Research Summaries episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.