Skip to content
Pop Quiz, AI: How Do You Test a Thinking Machine? episode artwork

Feb 12, 2025 6 min

Pop Quiz, AI: How Do You Test a Thinking Machine?

from Generative AI 101 Hosted by Emily Laird

AI keeps bragging about its "reasoning skills," but is it actually getting smarter, or just better at faking it? In this episode, we put AI’s so-called intelligence to the test with hardcore benchmarks—BIG-Bench HARD, TruthfulQA, and more—to see if these models can truly problem-solve or if they're just memorizing answers like a sneaky high schooler. Spoiler: Not all AIs are built the same, and some are way better at bluffing than thinking. Tune in to find out who’s the real deal and who’s just a smooth talker. Connect with Us: If you enjoyed this episode or have questions, reach out to Emily Laird on LinkedIn. Stay tuned for more insights into the evolving world of generative AI. And remember, you now know more about reasoning models than you did before! Connect with Emily Laird on LinkedIn

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Pop Quiz, AI: How Do You Test a Thinking Machine?

0:00 6:08

No transcript yet.

Requests help us decide what to transcribe next.

Share this episode

Similar podcasts

CISO Perspectives (public) N2K Networks This season on CISO Perspectives, host Kim Jones explores some of the challenges of leading through uncertainty. We explore the complexity of the changing nature of regulation and working with the federal government, the evolution of privacy and fraud, and how emerging technologies like AI and quantum computing are changing cyber. When you don’t know what questions to ask, you’re afraid to ask, or don’t know who to ask, CISO Perspectives provides the foundation for learning in this brave new world.

Frequently asked questions

How long is this episode of Generative AI 101?

This episode is 6 minutes long.

When was this Generative AI 101 episode published?

This episode was published on February 12, 2025.

Can I download this Generative AI 101 episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.