[EN] Jean Zay Supercomputer, Large Language Models - Nathan Cassereau, Hatim Bourfoune

I met with Nathan Cassereau and Hatim Bourfoune from IDRIS, a national computing centre for the CNRS (the national research centre in France). Nathan and Hatim work on the Bloom project, an open source large language model, which was created using the Jean-Zay supercomputer. Thanks to Nathan and Hatim I had the chance to take a look at the machine after our interview. LLMs and AI/ML in general have created a lot of excitement. Hatim said he got into AI/ML himself, and he highlighted a Coursera course run by Andrew Ng. Here are a few links:https://arxiv.org/abs/2211.05100 a paper on BLOOM on ArXivhttps://github.com/ncassereau-idris/lm-evaluation-harness Evaluation of LM https://github.com/dptrsa-300/start_with_bloom Getting started with BLOOM on GitHubhttps://huggingface.co/bigscience/bloom Summary on BLOOM from Huggingface https://www.technologyreview.com/2022/07/12/1055817/inside-a-radical-new-project-to-democratize-ai/ a technology review on BLOOM by MIThttps://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32 another BLOOM articlehttps://www.youtube.com/@CNRS-FIDLE YouTube channel by CNRS https://github.com/NVIDIA/Megatron-LM Megatron LM library used in the projecthttps://github.com/microsoft/DeepSpeed DeepSpeed library used in the projecthttps://pytorch.org PyTorch library https://www.genci.fr/en a national infrastructure to provide access to HPC (Grand Equipement National de Calcul Intensif) in Francehttps://en.wikipedia.org/wiki/Jean_Zay brief summary of Jean Zay's lifehttp://www.idris.fr/eng/jean-zay/jean-zay-presentation-eng.html The Jean Zay supercomputer at IDRIS/Paris-Saclay Get in touchThank you for listening! Merci de votre écoute! Vielen Dank für´s Zuhören!Contact Details/ Coordonnées / Kontakt:Email mailto:[email protected] RSE Slack (ukrse.slack.com): @code4thought or @piddie Bluesky: https://bsky.app/profile/code4thought.bsky.socialLinkedIn: https://www.linkedin.com/in/pweschmidt/  (personal Profile)LinkedIn: https://www.linkedin.com/company/codeforthought/ (Code for Thought Profile)This podcast is licensed under the Creative Commons Licence:  https://creativecommons.org/licenses/by-sa/4.0/