Skip to content
Greg Kamradt: Benchmarking Intelligence | ARC Prize episode artwork

Jun 24, 2025 48 min

Greg Kamradt: Benchmarking Intelligence | ARC Prize

from Agentic Conversations (formally mlops.community) Hosted by Demetrios

What makes a good AI benchmark? Greg Kamradt joins Demetrios to break it down—from human-easy, AI-hard puzzles to wild new games that test how fast models can truly learn. They talk about hidden datasets, compute tradeoffs, and why benchmarks might be our best bet for tracking progress toward AGI. It’s nerdy, strategic, and surprisingly philosophical. // Bio Greg has mentored thousands of developers and founders, empowering them to build AI-centric applications. By crafting tutorial-based content, Greg aims to guide everyone from seasoned builders to ambitious indie hackers. Greg partners with companies during their product launches, feature enhancements, and funding rounds. His objective is to cultivate not just awareness, but also a practical understanding of how to optimally utilize a company's tools. He previously led Growth @ Salesforce for Sales & Service Clouds in addition to being early on at Digits, a FinTech Series-C company. // Related Links Website: https://gregkamradt.com/ YouTube channel: https://www.youtube.com/@DataIndependent ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Greg on LinkedIn: /gregkamradt/ Timestamps: [00:00] Human-Easy, AI-Hard [05:25] When the Model Shocks Everyone [06:39] “Let’s Circle Back on That Benchmark…” [09:50] Want Better AI? Pay the Compute Bill [14:10] Can We Define Intelligence by How Fast You Learn? [16:42] Still Waiting on That Algorithmic Breakthrough [20:00] LangChain Was Just the Beginning [24:23] Start With Humans, End With AGI [29:01] What If Reality’s Just... What It Seems? [32:21] AI Needs Fewer Vibes, More Predictions [36:02] Defining Intelligence (No Pressure) [36:41] AI Building AI? Yep, We're Going There [40:13] Open Source vs. Prize Money Drama [43:05] Architecting the ARC Challenge [46:38] Agent 57 and the Atari Gauntlet

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Greg Kamradt: Benchmarking Intelligence | ARC Prize

0:00 48:30

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Agentic Conversations (formally mlops.community)?

This episode is 48 minutes long.

When was this Agentic Conversations (formally mlops.community) episode published?

This episode was published on June 24, 2025.

Can I download this Agentic Conversations (formally mlops.community) episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.