9/10 - Distributed Memory and Parallel Computing

Marc Hartung and I will discuss distributed memory in parallel computing in this episode, with tools like OpenMPI. We also discuss some of the hardware aspects around HPC systems and how shared and distributed memory computations differ.Links:https://www.open-mpi.org OpenMPI homepagehttps://docs.open-mpi.org/ the docs for OpenMPIhttps://www.mpi-forum.org The MPI Forum (who write the MPI standard)http://openshmem.org/site/ OpenSHMEM https://en.wikipedia.org/wiki/Distributed_memory summary page on distributed memoryhttps://en.wikipedia.org/wiki/InfiniBand InfiniBand network solution https://www.nextplatform.com/2022/01/31/crays-slingshot-interconnect-is-at-the-heart-of-hpes-hpc-and-ai-ambitions/ Slingshot network solution https://en.wikipedia.org/wiki/Partitioned_global_address_space https://www.techtarget.com/whatis/definition/von-Neumann-bottleneck the bottleneck named after John von Neumannhttps://en.wikipedia.org/wiki/Floating_point_operations_per_second overview of FLOPS (floating point operations per second)https://www.openmp.org/wp-content/uploads/HybridPP_Slides.pdf OpenMP and OpenMPI working together in a hybrid solutionhttps://blogs.fau.de/hager/hpc-book Georg Hager/Gerhard Wellein book on HPC for scientists and engineers Don't be shy - say HiThis podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK. Producer and Host: Peter Schmidt