GPU versus CPU: What is engineering really doing for us

Share Episode                  We sit down with Jaikumar Ganesh, Head of Engineering at AnyScale, to explore the intricacies of heterogeneous compute. He unpacks the growing CPU/GPU divide, detailing how ML pipelines require precise orchestration — using CPUs for data reading and writing while leveraging expensive, massive-die GPUs for chunking and embedding.         Warren brings the insight that, with AI agents rapidly changing how software is created, building is now a requirement of the business-focused team. And our guest shares how sales and marketing departments are increasingly using tools like Cursor and Claude to develop their own workflow automations. We discuss the challenges that this shift begs: what is engineering really doing for us?         JK emphasizes that the core responsibility of the engineering organization is reliability. While anyone can generate code, running stable production software requires the deep "battle scars", robust observability, and meticulous release processes that only a dedicated engineering team can provide.         That results in needing to find the right talent. But, finding the talent to maintain this critical infrastructure isn't easy, which is why JK advocates for highly creative hiring strategies. He shares incredible success stories of bypassing traditional recruiting by running hiring ads in foreign-language movies at local movie theaters and setting up booths at social food festivals to find uniquely qualified candidates.         🎯 Picks:         Warren - Archer's Don't Fire VolleysJK - Book: The Explorer's Gene