Inference: AI’s Hidden Engine

Nikola Borisov, CEO and co-founder of Deep Infra, joins the show to unpack the rapid evolution of AI inference, the hardware race powering it, and how startups can actually keep up without burning out. From open source breakthroughs to the business realities of model selection, Nikola shares why speed, efficiency, and strategic focus matter more than ever. If you’re building in AI, this conversation will help you see the road ahead more clearly. Key Takeaways • Open source AI models are advancing at a pace that forces founders to choose focus over chasing every release. • First mover advantage in AI is real but plays out differently than in consumer tech because models are often black boxes to end users. • Infrastructure and hardware strategy can make or break AI product delivery, especially for startups. • Efficient inference may become more important than efficient training as AI usage scales. • Optimizing for specific customer needs can create significant performance and cost advantages. Timestamped Highlights [02:12] How far AI has come — and why we’re still under 10% of its future potential [04:11] The challenge of keeping pace with constant model releases [08:12] Why differentiation between models still matters for builders [14:08] The hidden costs and strategies of AI hardware infrastructure [18:05] Why inference efficiency could eclipse training efficiency [21:46] Lessons from missed opportunities and unexpected shifts in model innovation Quote of the Episode “Being more efficient at inference is going to be way more important than being very efficient at training.” — Nikola Borisov Resources Mentioned DeepInfra — https://deepinfra.com Nikola Borisov on LinkedIn — https://www.linkedin.com/in/nikolab Call to Action If you enjoyed this conversation, share it with someone building in AI and subscribe so you never miss an episode. Your next big idea might just come from the next one.