Skip to content
Titans: Long-Term Neural Memory for Scaling Context episode artwork

Dec 6, 2025 15 min

Titans: Long-Term Neural Memory for Scaling Context

from Steven AI Talk Hosted by Steven

The provided text details the development of Titans, a new family of neural architectures designed to overcome the fixed context window and computational scaling issues inherent in modern Transformers. The core of this system is a novel neural long-term memory module that operates as a meta in-context learner, dynamically updating its weights based on a calculated "surprise" metric that includes both momentum and an adaptive forgetting mechanism. This mechanism allows the module to store persistent, historical information, thereby acting as long-term memory while the attention component handles immediate context as short-term memory. The paper presents three distinct architectural ways to integrate this new component: Memory as a Context (MAC), Memory as a Gate (MAG), and Memory as a Layer (MAL). Empirical testing across diverse tasks, including language modeling and extreme-length reasoning benchmarks, demonstrates that Titans achieve superior performance and efficiency compared to baseline models, successfully scaling to context window sizes exceeding two million tokens.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Titans: Long-Term Neural Memory for Scaling Context

0:00 15:59

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Steven AI Talk?

This episode is 15 minutes long.

When was this Steven AI Talk episode published?

This episode was published on December 6, 2025.

Can I download this Steven AI Talk episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.