#77- Ring Attention and 1M context window, is RAG dead?

An episode of the Life with AI podcast, hosted by Filipe Lauar, titled "#77- Ring Attention and 1M context window, is RAG dead?" was published on March 7, 2024 and runs 12 minutes.

March 7, 2024 ·12m · Life with AI

0:00 / 0:00

Summary

Hello guys, in this episode I explain how we can scale the context window of an LLM to more than 1M tokens using Ring Attention. In the episode, I also discuss if RAG is dead or not based on these advancements in the context window. Paper Lost in the Middle: https://arxiv.org/pdf/2307.03172.pdf Gemini technical report: https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf Paper Ring Attention: https://arxiv.org/pdf/2310.01889.pdf Instagram of the podcast: https://www.instagram.com/podcast.lifewithai Linkedin of the podcast: https://www.linkedin.com/company/life-with-ai

Episode Description

Hello guys, in this episode I explain how we can scale the context window of an LLM to more than 1M tokens using Ring Attention. In the episode, I also discuss if RAG is dead or not based on these advancements in the context window.

Paper Lost in the Middle: https://arxiv.org/pdf/2307.03172.pdf

Gemini technical report: https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf

Paper Ring Attention: https://arxiv.org/pdf/2310.01889.pdf

Instagram of the podcast: https://www.instagram.com/podcast.lifewithai

Linkedin of the podcast: https://www.linkedin.com/company/life-with-ai

Share this episode

Similar Episodes

New Season, Same God

Jan 15, 2025 ·15m

Standing On Business

Jan 15, 2025 ·18m

I’m Tired Of This Ship”

Dec 8, 2023 ·20m

Episode 26 - community energy & shared ownership

Oct 25, 2023 ·18m

Somebody or Nobody

Oct 21, 2023 ·19m

It’s My Mentality

Sep 16, 2023 ·18m

Similar Podcasts

Life with Rev. Q Quinton Steverson Daily conversations filled with inspirations, insightful thought and in depth prospectives aimed to motivate and challenge you in this journey called life. Ireland Creates with Aisling O' Rourke Aisling O' Rourke Ireland Creates with Aisling O' Rourke is a podcast about Ireland's storytellers. The story behind storytelling. In each episode, Aisling speaks with a different creative about their life, creative practice and what storytelling means to them. Ireland Creates has featured Writers, Photographers, Musicians and Filmmakers to name just a few. If you would like to get involved please check out www.irelandcreates.com or email Aisling on [email protected] Leanne Wood Podcast Leanne Wood With more than twenty five years experience as a political activist on the Welsh left in various roles, including the leadership of Plaid Cymru and Member of the Senedd for the Rhondda, I am now seeking to contribute to political life with a podcast.This new platform aims to give voice to those people and issues we don’t hear enough from or about in the mainstream media. Our politics needs a different conversation. I hope that this podcast will help with that. Confession Session With Life Coach Francis Camia FRANCIS CAMIA This podcast focuses on life's confessions and discussions that will hopefully make an impact to everyone and lead to a better transformation of themselves. We will be talking about purpose, relationships and anything under the sun. It also aims to shed some hope and positivity to all the listeners. On a personal note, you may also send a message or if you want to have a personal catch up or a collaboration. Let's talk!

URL copied to clipboard!