Machine Learning Engineering Is the Ultimate RL Task | Jennifer Zhao

Jennifer Zhao, co-founder and CEO of Preference Model, discusses coming out of stealth with a $16M seed raise led by Andreessen Horowitz and open sourcing Karrot. She explains why training models on RL environments to perform machine learning engineering and alignment is the highest-leverage path to safe AI development. Check out our sponsor Lovable: https://lovable.dev. Turn ideas into software people love. MTS is a live news and interview show covering technology, business, politics, and culture as it happens. Follow MTS: X | ⁠https://twitter.com/MTSlive⁠ YouTube | ⁠https://www.youtube.com/@mtsituation⁠ Spotify | ⁠https://open.spotify.com/show/4HUNNmV1pjV7CW6Lphm1rM⁠ Apple | ⁠https://podcasts.apple.com/us/podcast/mts/id1891088763⁠ Substack | ⁠https://mtslive.substack.com/⁠ Interested in sponsoring the show? ⁠⁠⁠⁠⁠⁠[email protected]⁠⁠ -- Brought to you by: Mongo DB - Anyone can prototype an agent. Learn how to get them into production on September 30th in New York - register at https://www.mongodb.com/events/mongodb-local/nyc Lovable - Turn ideas into software people love with Lovable. Start today at https://lovable.dev/ Larridin - Finally prove what your AI investment is worth. Head to https://larridin.com/ today to start measuring your AI’s bottom line. Neon (by Databricks) - Get up to $100K in credits for your startup at ⁠https://neon.com/mts⁠ ElevenLabs - AI that communicates at human level across every channel and modality. https://elevenlabs.io/mts Arena - Measuring AI performance in the real world. ⁠https://arena.ai/⁠ Kong - The AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. https://konghq.com Today’s stream is sponsored by VCX, the public ticker for private tech. Head over to http://getvcx.com to learn more Blitzy - Autonomous software development for enterprise codebases. Ship 5x faster. https://blitzy.com AdQuick - Make your brand a billboard. Out-of-home advertising as easy to scale as digital. https://adquick.com Chargebee - Billing and monetization infrastructure for the AI era. https://www.chargebee.com/ -- Timestamps: (00:00) Introduction (01:32) Origin Story of Preference Model (02:42) AI Performing Machine Learning Research (03:59) Considerations for AI Alignment (04:55) Preventing Reward Hacking in Models (06:23) Why Environments Need Depth (07:45) Bottlenecks in Recursive Self-Improvement (10:33) Predictions for AI Research Progress -- Note: This podcast is not investment, legal, or tax advice, and is intended for informational and entertainment purposes only. Hosts and guests may hold positions in the companies and securities discussed; do your own research before acting on anything you hear.