#028: Nazneen Rajani – Building GPT from scratch, Hugging Face, AGI, open source AI

Nazneen is a Research Lead at HuggingFace, a startup with a mission to democratize ML, leading the LLMs trained using RLHF direction. Before HF, she worked at Salesforce Research where she led a team of researchers focused on building robust natural language generation systems based on LLMs. Her expertise lies in training and evaluating LLMs, focusing on interpretability, robustness, factuality, and commonsense reasoning. She completed her Ph.D. in CS at UT-Austin. Nazneen has over 50 papers published at ACL, EMNLP, NAACL, NeurIPs, and ICLR and has her research covered by Quanta magazine, VentureBeat, SiliconAngle, ZDNet, and Datanami. In this episode, expect to learn why open sourcing AI might be the future, what GPT actually stands for, how text tokenization actually works, how long it will take to get to AGI, the challenge with building effective AI models, advice for people getting into AI, and much more... #chatgpt #huggingface #openai GUEST LINKS: —————————Website: https://www.nazneenrajani.com/ Twitter: https://twitter.com/nazneenrajani LinkedIn: https://www.linkedin.com/in/nazneenrajani/ CONNECT WITH ME: ————————————LinkTree: https://linktr.ee/barrywinata LinkedIn: https://www.linkedin.com/in/barrywinata/ Twitter: https://twitter.com/barrywinataInstagram: https://www.instagram.com/the.prompt_/ YouTube: https://www.youtube.com/@barrywinata RSS: https://api.substack.com/feed/podcast/448257.rss Patreon Support: https://www.patreon.com/barrywinataTIMESTAMPS:————————————(0:00) Intro (1:21) Nazneen's role at Hugging Face (5:29) What is Hugging Face? (10:05) Work culture at Hugging Face (14:44) Nazneen's journey into AI (19:10) The importance of learning AI fundamentals (21:21) The emergence of AI in India (23:27) Building AI models (26:47) The value of good model inputs (29:41) Model robustness (33:51) Scaling models and emergent properties (37:10) Explaining GPT (40:31) Understanding text tokenization (42:19) 'Attention is All You Need' (45:25) Why ChatGPT is so powerful and what it means for AI (49:15) Halting AI progress and regulation (54:32) Closed source vs. Open source AI (58:06) AI concerns (59:32) AGI - Artificial General Intelligence (1:01:50) Advice for people getting into AI (1:06:14) Prompt Q&A (1:10:52) Outro This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit promptpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit beesletter.substack.com