Hey, Bankless Nation, we've got some extra bonus content for you here on the podcast feed. This is an episode from Limitless, our sister podcast that focuses on AI and frontier tech. If you remember earlier this year, we were just doing a ton of AI content on Bankless just because AI is supremely interesting and worth paying attention to. But we were making so much AI content that it just started to crowd out all of the crypto content that we also wanted to keep doing. So we made a second podcast called Limitless and you should go subscribe to it. The episode from Limitless that you're about to hear right now is with Limitless host Ajaz and I as we interview two OpenAI team members who are in charge of memory and personalization of ChatGPT.
They're the ones who helped ChatGPT evolve from just a dumb LLM chatbot to your personal assistant who can remember everything about you and make guesses about your interests and just overall be helpful and perhaps be a friend rather than just an inert chatbot. It's not often that OpenAI team members go on podcasts. We are very privileged to be able to host Samir and Christina to get all of our questions answered about our favorite little chat assistant and what it's going to be able to do for us in the future. So I hope you enjoy this episode from Limitless. If you do enjoy it, make sure to click the link in the show notes to go subscribe to the Limitless podcast so you can stay up to date with the rapidly evolving world of AI. So let's go ahead and get right into this episode.
I'm so excited about today's episode. We are joined by Christina Kaplan and Samaya Ahmed, who both lead memory and personalization at OpenAI. Samaya, Christina, welcome. Thank you. So excited to be here today. Okay, you are both solely responsible for creating the most valuable moat that's ever been discovered in AI, memory and personalization. In fact, I remember the very moment you guys released the memory feature, because five seconds before that, I was ranting to my friend about how annoying it was that ChatGPT had no recollection of who I am when I opened a new chat. And it was like this light bulb moment went off where I stopped viewing ChatGPT as this tool.
and more of a friend. And that's an incredibly sticky product experience. But I want to hear about it from your side. What was it like to develop ChatGPT's memory feature? Give us the inside scoop. What was it like to go about creating this vision around memory? I think thanks for the generous introduction there, Jaz. I think we probably would classify it slightly different. And to be totally candid, memory in ChatGPT predates both Christina and me joining the team. I can rewind all the way back to 2022. ChatGPT comes out, and I'll give you a little bit of an analogy here that I think is a truly apt way of how we think about it.
And 2022, sort of like, imagine you had an assistant in a room, you walked in, you asked a question, and it answered natural language. And this was the first time you've seen a computer answer a natural language. And it gives you the answer, you leave the room, stops working, your assistant stops working. You walk back in, you say, hey, how's it going? Ask another question. Assistant does not remember who you are. No idea. It's a paradigm shift, but it really is not, you know, the type of experience that you would normally have in a situation where we're working with an assistant. So fast forward maybe to 2024, it's almost like we gave ChatChiefD a notebook, and it could write certain things down as you were talking to it. And you'd say something.
We'd try to write it down. We'd try its best to know what's going on. You'd leave the room. You'd come back. You'd ask another question. And it would, like, scour through the notebook to help answer the question it had. And it's a little bit like, you know, maybe the movie Memento, where you got tattoos on you and you have, like, some clues, but it's not a perfect picture. So, you know, Christina and I have been thinking, like, how can we make this better? And there are a lot of people involved here. But that's when, you know, the April memory update, that's around the time we came up with our latest variation of memory. Yeah, so the latest variation of memory, like, we had an assistant, a person that had a notebook with information about you, but still very limited. Like, might understand your name, but doesn't really understand you or who you are or really can't carry a conversation.
from new conversation to new conversation. So the update to memory that we launched in April was really trying to bring a more natural memory to Chachi BT. So taking this like person with, you know, memento style tattoos to be like a real person that remembers you, that can like pick up a conversation from where you left off. So that's really what inspired this launch in April is turning Chachi BT's memory into something that's more natural. That is the beginning of what you might expect from like a real assistant that is like the same person in the conversation time after time that you come back and talk to them. Yeah, and that's when I think, you know, that released early April. I mean, that's funnily enough how, you know, we got in touch.
because we'd listened to, I think, a podcast you've done on this. Yeah, it was a show. Actually, what was it like? So we were talking about memory. I think me and Josh were all very...
inspired just by what could happen next. What door is this unlocked? You know, in hindsight now as a ChatGPT user, I'm like, oh yeah, this seems like a very obvious like feature to unlock. But before it got delivered, I don't think anyone really knew where the arc of this product was going. Again, now it seems obvious. Maybe you could put us in your shoes as you were listening to us like chit chat about our experiences with memory. What was going through your guys' heads? Yeah, actually like Christina had just gone on vacation and I was just, you know, scrolling through everything on Twitter or looking for different coverage of it. And, you know, the change was not like a visual feature that you could see. And so not everyone picks up on it right away.
And I remember someone forwarded me a link to the podcast and I was a little skeptical and I was like, okay, let me, let me read, let me listen. Limitless has forked off from Bankless. I texted Christina. I said, you got to listen to these crypto bros talk about ChatGPT. I was skeptical. And then Samir basically like bullied me over the course of multiple days to listen to this podcast. He was like, this is like the thing you have to do. It was really interesting to hear you both talk about this launch and to hear, I think like David, you talk about it's a small step, but maybe it's a small step. That's really a big change.
And, I mean, Ajaz, we, like, DM'd you on Twitter immediately to talk because we were just so curious. Like, your reaction was, like, much stronger. But I'm very grateful to Samir for influencing me to listen to this on vacation. Also, funny story, Ajaz, you made us do, like, a three-factor auth. We DM'd you. And you're, like, can you prove this? Yeah. From our OpenAI accounts. And then I got a mutual friend three-factor authentication before Ajaz would talk to us. Yeah, I didn't believe OpenAI was listening to the Limitless podcast back then. But turns out we say some interesting things.
Yeah, the context for that is that, you know, in crypto, there's a lot of scams. And so when somebody from OpenAI texts you, messages you on a very public social media platform, you're like, what's going on here? Can we color in a little bit about what was the bigger, the broader reception of memory when it was introduced? Because I think we're just to kind of jog our listeners' memories, because that was, you know, so many months ago, half a year ago, and time flies. I was skeptical. I honestly can't necessarily remember what I was skeptical about. I know I remember Josh being very excited, but like, give us the range of like feedback and reception that ChatGPT users had to the memory feature. This was a change that was mostly under the hood, except we did release a...
notification to share with people that this change was happening. And it kicked people into a conversation that was something like, tell me based on everything you know about me, about who I am and like make it catchy. And I think the reception was twofold. There was like really strong reception, actually stronger than I think we had originally anticipated to that prompt. Like people were really excited to see how ChachiBT had been understanding them and like really learning about themselves from hearing about ChachiBT's perception of them.
But I think that we were surprised by the reception like it jazzes, honestly. Like, hey, this really changes the experience because other than that one notification, there really wasn't much to it visually. And we sort of hypothesized that this natural evolution of memory would be really meaningful. But it was a surprise to us how many people actually like felt that in their experience. And a lot of people will like extrapolate, come to, you know, jump to different ideas from that point in time as soon as they see it. And so it was really, really interesting to just see different takes on this. But, you know, to ground us in like how we think about it, a lot of it has to do with this just a universality of memory and how we can, we can sort of.
meet people where they are. And there's a little element to this around like, everyone has a different experience with ChatGPT. If you're, you know, living in San Francisco or if you're living in Indonesia, if you're in Brazil, everyone has a way that they communicate with other humans. And memory helps sort of like make the playing field universal so that people all over the world are able to like communicate with ChatGPT at the level that they understand. And it's not like mapping it to, you know, how people communicate with computers. Humans have been talking to humans for tens of thousands of years. Everyone sort of understands that. Humans have been talking to computers or interacting computers for 30 years, button clicks, like widgets.
And so that's maybe not as natural and not the best way to communicate and tackle higher order concept discussions. Hey, Josh, I'm curious to replay a bit of like your reaction because it was very strong and also like how you think about it now a few months later. Okay, so. How I thought about ChatGPT pre-memory was it was like talking to a friend that had amnesia. You'd have a conversation, you'd get into a topic about something, and then a couple hours later, it'd be like, hey, who are you? Tell me about yourself. And that kind of broke the illusion. Maybe illusion's not the right word, but it kind of broke the experience a little bit. And so when I saw that memory update on my app, what you described as a kind of small internal under the hood update meant a lot more to me because I knew that over the next few days, I'd be having less of those, hey, this is me conversations and more of...
Just conversations that flowed into something much more bigger, that compounded. When you guys think about ChatGPT and the evolution of the product, I'm a psych major, and I find it very easy to put into metaphors a lot of how ChatGPT works, or at least how I understand it to work. So like the short-term, long-term memory. Like the short-term memory is literally what is the last four prompts that the user and I have just been discussing. So let's retain that information first. But also in my context and my response, let me also understand a little bit about the deep truth that I know about this user.
and their interests and their hobbies. And I will frame my response based on like the long-term memory that I understand the user to be. When you guys are thinking about Chattoot, do you think of it in a cognitive psychology lens? Yeah, we look at it. I think, you know, we try our best to look at some of the prior art here. And in that scenario, it's really helpful to understand like how humans interact with humans and some of the advantages and pitfalls there. And Chattoot's memory is today nowhere as good as human's memory and sort of understanding the gist of people, interactions, the fidelity and the triggers. So there's a lot that we can draw on from the sort of existing cognitive presence.
it's like a much more natural way for people to interact with something as well. Like on the other end of this is a person. So a person, like I am interacting with chat to PT. I interact with like many other people. It's much more natural to interact with something that you expect to be an assistant in a way that you would interact with another person. So I think a lot of the, like at the end of the day, this really comes back to like not chat to PT, but the user. Like how do people expect something to interact with them? And like, how do we meet everyone where they are and like what they expect as the user? I can't exactly remember what I was so skeptical about. Clearly that skepticism has dissolved because I remember just maybe one, two weeks after this feature got released that I realized like, oh.
this product gets better the more I tell it about myself. A lot of people had reached out to us further down the line, you know, weeks, months, and then mentioned that this is one of their favorite features. And, you know, I would always ask, is it a single moment, a magic, or is it a slow burn? And, you know, different people give different responses, but for some people it's a slow burn. Like, they just realize that things have shifted and, like, ChatGPT is behaving.
in a way where clearly it understands them. And I think it has to do with maybe how you're using it. Everyone uses a different fashion. Yeah. And this feature, the memory feature unlocked, I think, what is many people's, not favorite prompts, but I think a prompt that many people have asked ChachiPT, which is based on everything that you know about me, who do you think I am? Or what do you think my personality is? Or some like hold up a mirror to me and tell me what you think I am. And so it kind of turns ChachiPT into a diary. In a sense, it's like I have like queries and prompts, like help me plan a trip that I want to make. But also the trip that I want to make tells ChachiPT a little bit about myself. And so every single prompt, it...
It's like a little bit of just like, here's one more shade of who I am. And it really encouraged me to like, you know, give Chachapiti all my data, tell me all my interests, tell me the secrets that I've never told anyone before. Yeah, talk to us about the arc of how user behavior changed with the introduction of the memory feature. It's a good question. I actually would say my user behavior has changed much more with, well, I think both with enhanced memory, but also pulse, which came out last week, which is kind of like natural evolution from memory as well. But I think with memory, like I started sharing with Chachapiti more context. So instead of just going there with a question that I wanted an answer to immediately, like I knew that this context.
would come up in like future conversations with Chachapi Tea. So if you share like, hey, I'm looking for a fish forward restaurant for tonight, like help me find that. It's kind of unclear to Chachapi Tea. Like, why am I looking for that? Am I looking for it? Because like I'm pescatarian. And so it shifted my behavior into being more along the lines of like, hey, I'm pescatarian. I'm looking for a restaurant tonight. Can you like help me find something? And like I've started sharing facts along the way, as opposed to just like going in with a question like I would in like a traditional search product. And then like down the line, as Samir mentioned, it's like a slow burn. You see those facts come up in the future. And so now Chachapi Tea, if I'm like asking about a restaurant, it will suggest like.
fish forward pescatarian restaurant friendly restaurants and and i can just like expect that now and so i've started just sharing like facts about myself in context and i like expect to be helpful in the future it seems that this this approach with pulse and with the data you guys are amassing is solely meant to kind of help a better user experience for the person using chat gpt right it's like this personalized experience that helps them improve and get better but i do also want to address the elephant in the room which is you guys now own so much data and not just any data very personal data i tell chat gpt everything and very willingly very consentingly and i have to ask what are the plans with this massive honeypot of data? Are there any kind of like thoughts on new products or maybe new experiences that you could do with something like this? Yeah, I mean, I think we're very focused on like helping people achieve really meaningful things in their life.
And definitely, you know, Sam has said too, like Chachapiti will probably be the most like sensitive account that people have. And like we take this very, very seriously, but there's also so much to be gained for this. Like I have many personal experiences in the health domain. For example, I was going to get vaccinated for a trip I went on in July and I had already shared some of my prior like vaccination forms as well as my labs that I had done with Chachapiti. And I asked the nurse that I went to go get vaccinated with like, hey, what vaccinations do I need to go to this trip in like a month or so? And he gave me four vaccinations I needed. And then I also asked Chachapiti like just to check and see.
what Chachubiti thought, and Chachubiti gave me five vaccinations that I needed. I was like, what is the difference here? And the nurse had looked at my prior vaccination forms, and Chachubiti had pulled out that I had low immunity to varicella from a lab I had done early last year, and was like, based on your vaccination forms, you have these four that you need, and then you also have low immunity to varicella, so you need to add that one as well. And I think that's an example of such a sensitive use of my data. I have literally uploaded all of my labs and vaccination forms to Chachubiti, but...
It's helped with such a like individualized outcome. I wouldn't expect anyone to be able to go look at every single form I've ever shared with them before for every single like doctor's appointment. And so now I know that I can like go to ChachiBT and ask like, hey, I'm going to this doctor's appointment, like what's relevant for that. But at the same time, like I am a user of ChachiBT sharing my like health information with ChachiBT. And I think we take that like very, very seriously and want to make sure that people are able to get a lot of value out of ChachiBT in the way that they want. But yeah, at the same time, like really taking data privacy and like.
security really, really seriously in the work that we do. So I could imagine some property, some thing, like sign in with ChatGPT on this website. And I go to some non-open AI website and I sign in with ChatGPT. And now it's my agent, my LLM, my chat, that is piped into this website. My identity signed in with this website. And all of a sudden, this website can query. It's like, hey, what does this user want to see? And my LLM could answer them. These are just my thoughts about what you could do. But there is some notion about the data and knowledge that ChatGPT knows about me could be extended to the broader internet. And the internet can finally be truly personalized to my interests.
When you guys think about the product, how over the mark am I? I think that, you know, if you look at it back to our like assistant intern analogy, interns in the room, you go in, you talk about some stuff. You mentioned the, you know, in my case that like I recently had a son who's nine months old now and I come back tomorrow and the assistant's like, oh, hey, like, how's it going? Maybe here's a tip around like what you should expect in terms of like the development cycle and like when they're speaking or they're babbling, this is what it means. If we extrapolate that, you know, and we just keep following that arc, I would love for that assistant to be able to go into the real world and, you know, do things on my behalf or represent me on my behalf to solve my goals. And it's just, you know, today Chachu ET as a property can deliver so much value. But if you look at the arc of this, there's things that happen all over the internet. And I think like aspirationally, if you had an assistant, you'd want them to be able to go and solve problems all over the internet. Talk to me about how you align an AI towards a human's goals though, right? Like when I think about like...
When I think about being a new person, like making a new friend or learning deeper about a person, it takes so many different types of inputs. I'm looking at the expressions on their face, I'm listening to the tone of their voice, and then I'm listening to the actual words that they're saying, not to mention that they have a lot of past experiences that influence the words that actually come out of their mouth. When I think about the chat GPT relationship, I'm just, I'm slamming a bunch of letters. Sometimes they make no sense. Sometimes there are a lot of typos. How do you process that into an AI and say, this is the thing David needs to do, or this is the thing Ejaz needs to build?
Well, we are at an amazing research lab. So one thing that's been really special about the personalization team, too, is we have like an end-to-end research product team. So we're able to make progress really quickly on the research side based on what our goals are in the product. And we have, as Samir said at the beginning, it really like takes a village to make personalization happen. And we have some amazing researchers working on this problem. And it works the other way, too. From the product end of it, if you pull out Chatubet Pulse today, you know, you get to the bottom of the page and it will say, you know, curate for tomorrow. And that's effectively, you know, a goal that you could set for tomorrow or something that you want for tomorrow. I mean, it doesn't have to be for tomorrow necessarily. You could set a longer term.
aspect like i mentioned like oh keep me up to date with developmental milestones or it could be somewhat you know trivial like i'm a formula one fan and these races happen like at absurd times in the morning and i'll always say like keep me up to date with formula one news but do not spoil the race for me and otherwise i'll be very angry and there's no like that's there's no way you could communicate that in any other tool that has existed prior to chat and get that level of like you know captures level of nuance to it and provides me utility in that case yeah i already am doing this with like the curate function and impulse as as samir mentioned like i'll say hey i'm going to a trip next week. I'm like going on a trip next weekend. Help me plan for my trip. I'm like, that's a goal I have to like plan a great trip to like London for a wedding that I was just at. And ChatGPT is like sharing like coffee shops and pescatarian restaurants that I might like and helps me kind of like look ahead and see like what's upcoming in my life. And I'll just like start sharing things like this is my goal. This is what I'm doing and sort of expect ChatGPT to like meet me where I'm at and help me help me do that better. Let's get into Pulse. This is a brand new feature that Sam Altman announced and credited you both specifically as a next step in memory and personalization. I kind of think of it as like my own personalized ChatGPT.
I've gone into the habit every evening, actually, before I go to bed of talking to it. Samir, you just gave an example of saying, you know, tell me about the F1 news, but don't tell me who won because I want to watch the race. I've been doing similar things for all the other tropes of my life. You've explained kind of how it works, but let's pop open the hood. Like what's actually happening underneath that? You've explained memory and kind of like putting together the identity behind David, the identity behind Ejaz. Tell us about this next step of personalization and how it works. Well, yeah, first of all, this is like a feature that took a village and we have an amazing, amazing team that helped build Pulse. But it really is that next evolution of ChachiPT. So you have an assistant.
that you go into the room, you start talking to them, and now they know who you are every time you come back. But when you leave the room, they stop working. And so you have an assistant that basically like only works in your one-on-ones effectively. And like, that's not someone that I would hire to be my assistant. It's someone that might know me and understand me, but only do work when I'm there, that I generally have to watch them do. And with Pulse, we started to think about how can ChachiPT in understanding you, understanding what's important to you and your goals in life, help you when you're not there. So you don't have to spend all of your time in ChachiPT. And the goal here is not that people are spending all their time in ChachiPT. It's that actually we're more successful when ChachiPT is doing things and helping you and you don't have to be there.
So you come back every morning and there's like new value for you to get from your assistant that was like effectively doing work when you went there. So basically behind Pulse is like, we try to understand, you know, what's going to be important to you in the next few days. Upcoming, we introduced calendar and email connectors as part of Pulse as well. So not just looking at memory, but also, you know, what's upcoming in your calendar? What's just happened in your email? And how can ChatGPT like understand all this information, understand what's important to you, and then help you prepare for your day? It's important to note that, you know, it jives when you wake up and you scroll to the bottom of your Pulse page, you know, it ends. And so to sort of...
reiterate Christina's point there. We are trying to get you prepared for the day, help you accomplish whatever that goal is, whether it's a gym goal or it's a, you know, some sort of event you're planning for, and then, you know, get back to your day. And if we do that, then we can take the time in the background with ChatGPT, can figure out what's important and, you know, spend the rest of the time that you're not interfacing with it, getting work done, which is kind of what you'd want from most of your assistants. Yeah, you can kind of think of it as like while you're asleep, ChatGPT is like trying to process all this information you shared with it to understand how it can help you the next day. So then you wake up and there are like things that are ready for you that right now are mostly content-based. So like, what can we, what can your assistant share with you that's helpful? But you can imagine you wake up and your assistant's like, Hey, I wrote this email for you. Do you want to send it? Or like, Hey, I, you know, prepared this for you. Do you want to go do that? And so I think the goal here is like becoming more and more like helpful and actionable over time. Hey everyone, if you want to listen to the rest of Christina and Samir, I'm going to make you go over to the Limitless feed to do that. We're cutting you off halfway. There's a lot more good stuff left in this episode and also on the Limitless feed in general. So to listen to the rest of the episode and to also just stay ahead of the curve when it comes to AI and other world-changing tech, go subscribe to the Limitless podcast. There's a link in the show notes.