PODCAST · technology
Inside the Black Box: Cracking AI and Deep Learning
by Arshavir Blackwell, PhD
How do Large Language Models like ChatGPT work, anyway?
Episodes
-
Three Ways to Be Wrong About the Past Tense
A follow-up in the past-tense series testing whether U-shaped learning, model size, or measurement explains irregular verb behavior—and finding the language matters too.
-
Not at This Address
This episode challenges the old idea of a single language box in the brain, using aphasia studies and hospital control groups to show how language processing is more distributed than once thought. It also explores how different languages rely on word order or verb marking in different ways, and…
more -
My Advisors Argued This for Thirty Years. Now You Can Check
Thirty years ago, the authors of Rethinking Innateness argued that grammar could come out of a learner with no grammar built in. Nobody could check it. Now there's an instrument. A reading of the Inside the Black Box essay on what Elman, Bates, and Karmiloff-Smith would make of transformer…
more -
Inside the Past Tense: Raw Letters, Raw Sound
Can a model learn past tense the way children seem to, if you remove the tokenizer from the equation? This final installment tests raw letters and raw sound, probes whether a U-shaped pattern can be forced, and shows why the answer points to distributed, frequency-driven learning rather than a...
-
Forcing a Neural Network to Add -ed
Part 3 of 4. After two episodes arguing there is no discrete past-tense rule inside a language model, this one turns the strongest possible search on the question: gradient descent, hunting for the single internal direction that best forces goed over went. It works — 100% of the time, even on...
-
The Wug Test for AI
This episode explores a modern twist on Jean Berko’s famous wug test, comparing how large language models handle made-up verbs versus familiar ones. It digs into tokenization, internal feature probes, and why neural networks seem to approximate grammar through probabilistic patterns rather than...
-
Learning the Past Tense in AI
This episode revisits the famous debate over whether language is learned through symbolic rules or distributed neural patterns, using the classic irregular verb U-shaped curve as the battleground. It then compares child language development with training snapshots from modern AI models, showing...
-
Fine Tuning Lora: It's Not What You Think
When you fine-tune an AI model, what changes inside doesn't predict what changes outside. This week on Inside the Black Box, I break down why — and what it means for anyone auditing or regulating these systems.
-
When Fluent Answers Start Sounding True
This episode explores why smooth, coherent language can feel more credible than it is, and how processing fluency, familiarity, and authority cues shape what we believe. It also digs into why conversational AI is especially persuasive, from polished explanations to confident-sounding confabulations.
-
Why Your Brain Believes the Model
The Heuristic Loop You Can't Break from Inside
-
When Polished Answers Feel Finished
This episode explores fluency-as-validity: the way polished AI responses can make us feel like the work of judgment is already done. It also looks at why large language models are so effective at creating the sensation of clarity, and why mechanistic interpretability may be a way to push back...
-
What Seneca Teaches Us that Marcus Couldn't
716 features fire on both Seneca and Marcus Aurelius but stay dark for ad copy. The model learned Stoic philosophy, not just an author's style. Plus: why 'inert' features aren't all the same thing.
-
The Pattern Holds for Another Author
We trained a fresh LoRA on the letters of Seneca and ran the same analysis pipeline we used on Marcus Aurelius and advertising copy. Every structural finding replicated. The model organizes its adaptation into five clusters: one tight (features moving in lockstep) and four loose (features...
-
The Pattern Holds
We replicated our Marcus Aurelius findings at a new layer, then threw the whole method at 12 commercial ad copy styles trained into a single LoRA. The patterns held, and the new domain revealed something we couldn't have seen before: the model organizes its adaptations by register family, not by...
-
Cracking Open the Black Box
We opened the 65%. The features that resisted interpretation one at a time turned out to organize into five co-activation clusters with clear thematic identities and causal effects nearly ten times stronger than any individual feature. Second in a series with John Holman.
-
Inside a Fine-Tuned Language Model
A concise, single-segment episode of Inside the Black Box: Cracking AI and Deep Learning where Arshavir Blackwell explains, in one continuous narrative, what neural networks are, how their simple units combine into powerful systems, and how learning by backpropagation sculpts their behavior. This...
-
What Counts as Structure? From Harris and Elman to Today’s Neural Nets
This episode of Inside the Black Box: Cracking AI and Deep Learning tells the story of an unexpected convergence in the history of language and AI. In 1995, Peter Bensch noticed that Zelig Harris, a mid‑century structural linguist, and Jeff Elman, a pioneer of simple recurrent networks, had...
-
Building a House Without Blueprints: When Interpretability Tools Work — and When They Don’t
This episode of Inside the Black Box: Cracking AI and Deep Learning explores a new theoretical framework that unifies sparse autoencoders (SAEs), transcoders, and crosscoders — and what it tells us about when mechanistic interpretability actually works. We start by demystifying these tools and…
more -
I Told My LLM Not to Say "Empower"
In this episode of Inside the Black Box: Cracking AI and Deep Learning, Arshavir Blackwell, PhD, takes engineers and researchers inside the practical mechanics of LoRA, low‑rank adaptation methods that make it possible to fine‑tune multi‑billion‑parameter language models on a single GPU.
-
Beyond the Surface of AI Intelligence
This episode dives into why judging AI by behavior alone falls short of proving true intelligence. We explore how insights from mechanistic interpretability and cognitive science reveal what’s really happening inside AI models. Join us as we challenge the limits of behavioral tests and rethink…
more -
Unlocking BERTs Hidden Grammar
Explore how BERT’s attention heads reveal an emergent understanding of language structure without explicit supervision. Discover the role of attention as a form of memory and what it means for the future of AI language models.
-
Cracking the Code of AI Interpretation
Dive into how we naturally explain neural networks with folk interpretability and why these simple stories fall short. Discover the journey toward mechanistic understandability in AI and what that means for how we talk about and trust large language models.
-
Decoding GPTs Hidden Circuits
Explore how sparse autoencoders and transcoders unveil the inner workings of GPT-2 by revealing functional features and computational circuits. Discover breakthrough methods that shift from observing raw network activations to mapping the model's actual computation, making AI behavior more...
-
Decoding Attention and Emergence in AI
Explore how attention heads uncover patterns through learned queries and keys, revealing emergent behaviors shaped by optimization. Dive into parallels with natural selection and psycholinguistics to understand how meaning arises not by design but through experience in both machines and brains.
-
When Knowledge Battles Noise in GPT Models
Explore how GPT-2 balances fleeting factual recall with generic responses through internal competition among candidate answers. Discover parallels with human cognition and how larger models navigate indirect recall to reveal hidden knowledge beneath suppression.
-
Inside Circuits: How Large Language Models Understand
Dive into the world of neural circuits within large language models. In this episode, Arshavir Blackwell unpacks how transformer circuits, attention mechanisms, and high-dimensional geometry combine to create the magic—and limits—of modern AI language systems.
-
Hallucinations, Interpretability, and the Seahorse Mirage
This episode dives into why advanced language models still generate hallucinations, how interpretability tools help us uncover their hidden workings, and what the seahorse emoji teaches us about model and human reasoning. Arshavir connects groundbreaking research, practical business importance…
more -
How Transformers Stack Meaning Like Finnish Words
Explore how large language models build up meaning in ways strikingly similar to the layered grammar of Finnish. Arshavir Blackwell reveals why understanding Finnish morphology offers a powerful analogy for interpreting the compositional logic inside modern AI systems.
-
The Mandela Effect in AI: Why Language Models Misremember
Dive into how and why large language models like ChatGPT mirror the human Mandela Effect, reproducing our collective false memories and misquotations. Arshavir Blackwell examines the science behind errors in models and minds, and explores how new techniques can counteract these uncanny AI...
-
Bridging Circuits and Concepts in Large Language Models
How do millions of computations inside large language models add up to something like understanding? This episode explores the latest breakthroughs in mechanistic interpretability, showing how tools like representational geometry, circuit decomposition, and compression theory illuminate the…
more -
How Transformers Turn Words Into Meaning
Embark on a step-by-step journey through the inner workings of transformer models like those powering ChatGPT. Arshavir Blackwell breaks down how context, attention, and high-dimensional geometry turn isolated tokens into fluent, meaningful language—revealing the mathematics of understanding…
more -
Can Smaller Language Models Be Smarter?
Today we explore whether mechanistic interpretability could hold the key to building leaner, more transparent—and perhaps even smarter—large language models. From knowledge distillation and pruning to low-rank adaptation, we examine cutting-edge strategies to make AI models both smaller and more...
-
The Weird Geometry That Makes AI Think
Explore how large language models use high-dimensional geometry to produce intelligent behavior. We peer into the mathematical wilderness inside transformers, revealing how intuition fails, and meaning emerges.
-
Can We Fix It?
Arshavir Blackwell takes you on a journey inside the black box of large language models, showing how cutting-edge methods help researchers identify, understand, and even fix the inner quirks of AI. Through concrete case studies, he demonstrates how interpretability is evolving from an arcane art…
more -
Using Symbolic AI to Explain LLMs
Delve into the mysterious world of neural circuits within large language models. We’ll dismantle the jargon, connect these abstract ideas to real examples, and discuss how circuits help bridge the gap between machine learning and human cognition.
-
Peering Inside the Black Box
Mechanistic interpretability and artificial psycholinguistics are transforming our understanding of large language models. In this episode, Arshavir Blackwell explores how probing neural circuits, behavioral tests, and new tools are unraveling the mysteries of AI reasoning.
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
ABOUT THIS SHOW
How do Large Language Models like ChatGPT work, anyway?
HOSTED BY
Arshavir Blackwell, PhD
CATEGORIES
Loading similar podcasts...
Frequently Asked Questions
How many episodes does Inside the Black Box: Cracking AI and Deep Learning have?
Inside the Black Box: Cracking AI and Deep Learning currently has 36 episodes available on PodParley. New episodes are automatically indexed when they're published to the podcast feed.
What is Inside the Black Box: Cracking AI and Deep Learning about?
How do Large Language Models like ChatGPT work, anyway?
How often does Inside the Black Box: Cracking AI and Deep Learning release new episodes?
Inside the Black Box: Cracking AI and Deep Learning has 36 episodes. Check the episode list to see recent publication dates and frequency.
Where can I listen to Inside the Black Box: Cracking AI and Deep Learning?
You can listen to Inside the Black Box: Cracking AI and Deep Learning on PodParley by clicking any episode. We provide an embedded audio player for direct listening, and you can also subscribe via your preferred podcast app using the RSS feed.
Who hosts Inside the Black Box: Cracking AI and Deep Learning?
Inside the Black Box: Cracking AI and Deep Learning is created and hosted by Arshavir Blackwell, PhD.
URL copied to clipboard!