Skip to content
Why Frontier AI Evals Are Falling Behind | Vincent Chen episode artwork

Oct 8, 2026 14 min

Why Frontier AI Evals Are Falling Behind | Vincent Chen

from MTS Hosted by MTS

Vincent Chen, co-founder at Snorkel AI, announces an expansion of Snorkel's Open Benchmarks Grants program to $30 million to keep evaluation infrastructure in line with accelerating frontier model capabilities. He breaks down the program's three core pillars—funding new benchmarks, launching the Open Benchmarks Red Team, and establishing the Snorkel Research Fellowship—while addressing continuous benchmark maintenance and AI alignment. Check out our sponsor Lovable: https://lovable.dev. Turn ideas into software people love. MTS is a live news and interview show covering technology, business, politics, and culture as it happens. Follow MTS: X | ⁠https://twitter.com/MTSlive⁠ YouTube | ⁠https://www.youtube.com/@mtsituation⁠ Spotify | ⁠https://open.spotify.com/show/4HUNNmV1pjV7CW6Lphm1rM⁠ Apple | ⁠https://podcasts.apple.com/us/podcast/mts/id1891088763⁠ Substack | ⁠https://mtslive.substack.com/⁠ Interested in sponsoring the show? ⁠⁠⁠⁠⁠⁠[email protected]⁠⁠ -- Brought to you by: Mongo DB - Anyone can prototype an agent. Learn how to get them into production on September 30th in New York - register at https://www.mongodb.com/events/mongodb-local/nyc Lovable - Turn ideas into software people love with Lovable. Start today at https://lovable.dev/ Larridin - Finally prove what your AI investment is worth. Head to https://larridin.com/ today to start measuring your AI’s bottom line. Neon (by Databricks) - Get up to $100K in credits for your startup at ⁠https://neon.com/mts⁠ ElevenLabs - AI that communicates at human level across every channel and modality. https://elevenlabs.io/mts Arena - Measuring AI performance in the real world. ⁠https://arena.ai/⁠ Kong - The AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. https://konghq.com Today’s stream is sponsored by VCX, the public ticker for private tech. Head over to http://getvcx.com to learn more Blitzy - Autonomous software development for enterprise codebases. Ship 5x faster. https://blitzy.com AdQuick - Make your brand a billboard. Out-of-home advertising as easy to scale as digital. https://adquick.com Chargebee - Billing and monetization infrastructure for the AI era. https://www.chargebee.com/ -- Timestamps: (00:00) Introduction (02:09) Snorkel Research Fellowship (03:54) Open Benchmarks Red Team (05:02) Auditing Broken Benchmarks (07:32) Benchmarks in AI Alignment (08:17) Science of Empirical Benchmarking (09:48) Closing the Capability Gap -- Note: This podcast is not investment, legal, or tax advice, and is intended for informational and entertainment purposes only. Hosts and guests may hold positions in the companies and securities discussed; do your own research before acting on anything you hear.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Why Frontier AI Evals Are Falling Behind | Vincent Chen

0:00 14:12

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of MTS?

This episode is 14 minutes long.

When was this MTS episode published?

This episode was published on October 8, 2026.

Can I download this MTS episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.