HPC & AI Convergence Enables AI Workload Innovation - CitC Episode 183

In this week’s Conversations in the Cloud, we are joined by Esther Baldwin, Artificial Intelligence Solutions Architect at Intel. Esther talks about AI for Good and how she was inspired by her mother to make a difference in the world. She encourages young people entering the industry to have courage – to talk to experts and leaders, to learn and grow, and not to let them themselves be held back. Esther recommends following fellow Intel star Riva Tez at https://twitter.com/rivatez for inspiration on this. A Forrester study noted that pre-configured and verified IT infrastructure was a key strategy for addressing solution complexity. Esther notes that our work isn’t complete unless solutions are consumable by customers at scale. On the topic of HPC & AI converged clusters, there’s a perception that if you want to do AI, you must stand up a separate cluster, which Esther notes is not true. Existing HPC customers can do AI on their existing infrastructure with solutions like HPC & AI converged clusters. However, running three workloads – HPC, AI, analytics – together can be tough, with the main problem being that they all have their own software stack and libraries that are optimized for specific applications. With a shared infrastructure, it can be challenging to run these all at the same time. The queuing system creates difficulties and even the underlying file system is incompatible. This is where Intel® Select Solutions comes in. Intel Select Solutions help people leverage experience with a pre-configured path to get a faster time to value. Intel® Select Solutions for HPC & AI Clusters offers users a quick start for those in the HPC environment wanting to run AI workloads. There are two options for Intel Select Solutions for HPC & AI Clusters – an open source version with Magpie and Slurm and a commercially available version with Univa Grid Engine. Intel offers a family of Intel Select Solutions for HPC and AI. Building on the foundation of Intel® Select Solutions for Simulation & Modeling, customers can also utilize solutions for simulation & visualization and genomics analytics, in addition to AI solutions like BigDL on Apache Spark and AI Inferencing. Esther notes that the HPC & AI convergence is already a trend, with AI becoming part of a wide variety of workloads. This solution will enable new HPC and AI use cases, in addition to seeing lower total cost of operations, better cluster management, and stronger workload performance. More information on Intel Select Solutions and Intel HPC solutions is available at www.intel.com/selectsolutions and www.intel.com/hpc.