Welcome to this talk. I'm extremely confused. I think maybe more confused than I've ever been in my entire life. Probably definitely more confused than I've ever been in my entire life. And I want to talk about that because I think, I don't know, when you're confused, that's the start of something interesting. If you're trained as a scientist, that's like the thing you want to pay most attention to is that moment where you're confused. Like, oh, this is interesting. I don't know what's about to happen. I've been in technology for like my whole life and I feel like I've always had this advantage where it's like, oh, I understand what's happening. I actually kind of think I know what's gonna happen over the next five or 10 years. And I've used that to my advantage. Like I've planned my career around it. I've founded companies around it. You know, I've kind of gotten ahead of...
the trend and gotten involved at the right time in a lot of different places. And that's like really been great for me. I no longer feel that way though. Like I can't see five years into the future. I can see like three weeks or less. And I really don't know what's gonna happen. So I want to have a series of questions that hopefully help.
me figure out what's going on. Maybe it'll help you all figure out what's going on. Maybe not. Not all questions are useful. But I think asking good questions is extremely important to running a startup, running a research team, running your life. Whatever you're doing, I think we all need to stop sometimes and just ask questions. But I think the moment in time that we're going through right now with AI is extremely challenging and fast. And if there's ever a time to stop and ask questions, it's probably right now. All these are my own, obviously. I think we're required to say that. My day job is running an alignment research team at Anthropic. I've also been through YC. I've done multiple startups in my life. So maybe my perspective on like...
AI plus startups is useful for all of you thinking about startup life or AI life. Okay, that's enough of a preamble. Let's jump in. This is sort of like the main question I want to ask. Everything's changing.
how should that impact everything about my life? Honestly, should you even start a startup is a big question. But let's assume that you're starting a startup or you're running a startup already. How should it impact your strategy? How should it impact your product? How should it impact how you're building your team? These are big, open questions. And I think AI is probably going to change how you answer them today and probably differently tomorrow. And there's this kind of paradox that I've seen throughout my career running startups and talking to startup founders. Everyone always tells you focus is everything. That's the advantage that startups have. Focus, focus, focus. Big companies can't focus. That's why you can out-compete them and run circles around them. But despite the fact that focus is everything, the other truth of running a startup is you have to focus on everything.
right like it's your job to think about hiring and fundraising and product and strategy and go to market and just like everything right um and then all of a sudden in the middle of like a product launch someone on your team quits and someone else threatens to quit and it's just like it's madness and it's happening all the time this is just an extreme paradox but i think it kind of uh makes founders suitable to this biggest question that we're all facing as a society which is what's going to happen with ai and what should we be doing because founders are the people that have to just answer every single question always so i think it's a good positioning there's been sort of like common quote unquote common just like in the last six months um best practices like what do you think about AI for your product. And people say, oh, you should think about what happens over the next six months. Think about what the next foundation models are going to be able to do and make sure you're planning your product, anticipating what those capabilities are going to do. Don't plan for the capabilities of today. I think it's extremely valid advice. You should definitely take it seriously. But I want to up the ante a little bit and say, actually, you should be planning two years in advance because it's extremely likely that we will have AGI in the next few years. Maybe it's not two. Maybe it's three. But I think you should be planning your company and your strategy around this fact. And there's extreme uncertainty. So don't take it too seriously. Don't have literally a two-year plan. But if you're not thinking a little bit about how this is going to change everything.
from hiring to marketing to go to market, et cetera, I don't think you're doing your job as a founder. Okay, so that's really the theme here today is I wanna like go through a series of questions and take this lens of AGI arriving and also just take a lens of like what's changing near-term in AI over the next six months. I think there's been a lot of conversations around like, oh, actually the impact of AI is gonna be slower than we think. And the reason is that big companies suck and they have to like, they take a lot of time to buy things, right? They don't realize the trend and then, you know, the enterprise sales cycle is really slow. So all the Fortune 500 companies are just gonna take years and years to like digest all the SaaS products that you all might wanna be building. And I think that's actually extremely nearsighted because what's gonna happen, I think,
open question is the buy side, the enterprises, they're going to get armed with AGI or strong agents over the next couple years as well, right? And that's not just via the SaaS products that they might be building. It'll just be natively inside, right? Their teams are going to be using the next versions of LLMs to make buying decisions, right? And to figure out how to accelerate their adoption cycle. The force of AI is not just on this like product revolution that the startups are building. It's also on the buy side, right? And I think this is like an interesting, weird thing about AI is that it's going to rise, you know, the water rises and all ships rise with it. It's not just the startups. The incumbents benefit from AI too. And I think we're seeing that in other places too, where large enterprises sometimes aren't even going out and buying.
software from SaaS providers anymore they're like I can just like throw two people at cloud code and they'll build it and it'll be dedicated to the the capabilities that I need for my organization so buy side is going to be evolving really quickly and I think it's an open question what does that mean right I think another angle on this is you know when you have AI powered outbound sales we think about what that is going to do to the marketplace but there's going to be this in this buy side too of like receiving those sale calls from AIs and trying to parse what's going on so the dynamics aren't really clear how it's going to play out this is a kind of related question like is software going to fully commoditize is it even going to make sense to run a SaaS provider in like two years or three years or is it like going to be the case that enterprises really do just build all their software in-house because it's just one, you know, one prompt to cloud code, the next gen version of cloud code, right? And actually you just need in-house product managers and they're going to do everything for you. Like that's one real outcome. That can happen to the consumer side too. Like maybe consumers eventually are just like not downloading apps anymore. They're just building apps on demand for themselves. They don't even think about it that way. Like they don't even think about them as apps anymore. It's just, yeah, I want my phone to do something for me. I ask it and yeah, it made an app for me. Sure. What's the point of downloading an app at this point? So that's one outcome. But I think another outcome actually is the opposite. It's like maybe all of this automation on generating code makes it easier to just raise the quality bar extremely high. And sure you can make
like the equivalent of today's apps very easily just by prompting. But can you make an exceptional app tomorrow, the equivalent of a great team that's working with AI to raise the bar? I don't actually know the answer to this question, but I think it might be different depending on the vertical. And you should be thinking about this for your product. I think it's an interesting question too. If you can write software on demand, do you even make apps anymore? Or is it really on demand? We're taking cloud code and things like it to ahead of time figure out what a user needs and then build as much software as we need to support all those use cases. But if you can do code on demand, why not do it on demand? On the fly, here's this user. They're doing something in your app potentially. And the app realizes that the app can't support what they want. And on demand, you generate code for that user.
I think this is a really interesting pattern if you can make it work. But it raises all sorts of issues around trust. It's one thing to do a generative UI, where maybe you're changing the shape of the interface for the user. But real new behaviors would require going down to the database level, on the back end. And if you want your AI to be able to do that on demand, you better trust that AI can do its job. And obviously, right now, AIs are not trustable enough to make that happen. So I think trust is going to be a big theme over how these different questions play out. Similar question, what should UI look like? I think people talk about generative UI. It hasn't really happened yet, but I'm like.
bullish to see when it does happen. Is an on-demand UI the right interface, or is there something even more different that we haven't thought about? I think about this specifically in the context of multimodality, weaving together auditory and images and video and text, and what's the right input back from the user. As a user, sometimes I want to speak, sometimes I want to use touch interfaces. It's really contextual depending on whether I'm in a crowded area, et cetera, right? I think you want to meet the user where they are. What's the easiest thing, what's the easiest way for them to interface with your product right now? That's the question you're thinking about. And I think all this ties into this big question I have, which is we have all these products today that have great distribution, and people realize that we need to insert AI into it. All the major players are doing this, right? Slapping like a chatbot on.
slapping on some like agentic behavior or whatever right they're like retrofitting their products to try to enable something with ai right but that's like trying to like sprinkle pixie dust on something it's natural to think it's better to build a new product from the ground up that's ai native like that's that's like my startup mentality like new technology revolution you need to start from scratch if you want to build this right uh but that might not be true it actually might be the case that retrofitting existing products with the benefit of distribution wins and i think again it might not it might not be universal this might be vertical by vertical but i think this is a really important question that you should be asking yourself and like don't just have an opinion like figure out the causal mechanisms that allow you to validate your hypothesis because I think these types of questions will make or break different products. Okay, so that's a lot about like products, product strategy, but I think every part of a startup is like extremely important. The most important thing is your culture, is your team. Will team sizes get even smaller? I think like that's like the default that people are assuming. But I think similar to products where like you can retrofit or you can build from scratch, I think there's gonna be this question which is, are like AI native teams that were built from the scratch to be AI native, are they gonna have some advantage over large companies that are like downsizing and finding ways to make themselves more efficient with AI, right? Are there like team patterns where you kind of like just operate differently if you started from an AI native place?
And actually that might be different every six, 12, 18 months, right? Because the capabilities of AI are changing. So like an AI native company today might be different than what an AI native company tomorrow looks like in 12 months. You might be outdated if you're not like thinking about how to retrofit yourself. Like I said, I think trust is going to be extremely important, right? So like how does the security model change? Like I mentioned this already in the case of on-demand code where you want your LLM to be able to go all the way back down to the database layer and do something on demand for that customer, for that consumer. But you can't do that if you can't trust the controls that you have in place. And if you can't trust the capabilities of the model to do the right thing, right? I think about this also in the context of like assistance.
Like the whole point of an assistant is to be useful to the consumer, to that user. But we have all these walled gardens and you're starting to see like maybe you need different agents in different settings, right? That's not what you want. As a user, you want one agent for all of your things, right? So me, I want my personal agent and my professional agent at work to be able to work together, right? That raises all sorts of concerns. Like there's stuff that I do on my personal time or you do on your personal time that maybe you don't want your employer to know about, right? So how do you make sure that the information is segregated while still allowing those agents to collaborate in the right way? Those are hard questions. Can you do it, right? Can you get to the same level of usefulness or do you have to compromise somehow on some of these security aspects?
A deeper question about agents is, you know, we think about alignment as can we trust the AI? But the truth is, even if you have a perfectly aligned, you know, quote unquote, like intent aligned model, it's going to be used by a corporation or startup to build an agent for that user. And then the question is, can you trust the startup that's building the agent, right? Is this agent that they've developed for you actually acting on your behalf? If this is like an ad-based company and you're using this agent to search for like a, you know, new brand of shoes or whatever, right? Like, is it going to be biased? Is it going to be like pumping you towards one direction? That's not what you want as the user, right? But it might be what you want as the corporation.
And then how do we start even asking these questions, right? I think going back to the previous slide around like personal versus professional agents, I really care if I'm using that agent. I really care that it's operating on my behalf, right? If it's like secretly operating on behalf of my company and it's like optimizing things in my personal life that's better for the company than me, that's like extremely scary. And I think it gets more scary the more capable the models get. So I think it's important to start thinking about trust, right? Like how do we instill trust not just in the models, but in the agents and the companies that are building these agents, right? And I think ultimately what that really comes down to is trust in you all, trust in the companies that you're building, right? How can a user trust?
this like company that's building this product especially you know already today we have these extremely small teams tomorrow the teams might be even smaller you might have these like semi-automated teams almost how can you trust that team you might say well i don't know like we trust companies today one of the core reasons we trust companies today is because they're composed of a diversity of people you can trust to some extent that if there's reasonable culture at a company that if the company decides if the ceo decides to do something bad that someone in the company is going to raise their hand and say no i don't support this i'm going to whistleblow i'm going to leak this i'm going to quit i'm going to take a bunch of people with me i'm pissed i hate this right and without the people supporting the company the company doesn't have a product right but in a semi-automated world it's no longer true and it could be the fact that a single person could make a decision that changes the entire impact of a product and there's no single person that might be aware of that except themselves it gets extremely easy for bad actors to do bad things and if you've followed along the history of silicon valley or the history of humanity the truth is a huge majority people are misaligned right especially when money is on the line So I think this is a really important question. And the truth is, we already think about this. Large enterprises, for example, already distrust startups, partially for this reason. There's a lot of reasons why large enterprises distrust startups. One of them is they're worried they're going to go out of business tomorrow, right? But it's just a lot easier for a small startup to do the wrong thing compared to a big company, right? That's part of what makes small startups successful sometimes. So I think this is already on the top of minds of enterprises, but it'll increasingly be on the top of minds of everyday people. So that kind of leads me to this next question, which is, well, how do we instill trust? What does a new set of guardrails need to look like? If you don't have all these human guardrails inside of your company where you've, you know...
worked really hard to build great culture and build a collection of people who care about doing the right thing, if that's not the thing that makes a company ethical anymore, how else can we make sure that there are guardrails? Like, what does that look like? A lot of people have been throwing around ideas on this. There's like this notion of AI-powered auditing. Like, what should an audit look like in an AGR world? There's this huge advantage that AI has over humans when it comes to bringing comfort to an audit, which is that they can be less biased, and they can also have no memory. For example, if you agree as a company to let an AI audit you, and you can say, hey, if you don't find any malfeasance, if after your audit you've decided that we didn't do anything wrong, as we claimed, maybe a public mission statement that the AI is confirming, then the AI deletes itself, right? All of its notes, et cetera, get deleted.
And that's a big advantage over having human auditors where they could take information with them, right? There's a lot of, there's already auditing happening today, right? For legal reasons, for financial reasons, you know, or you want to be like certified organic or whatever it is, right? But you have to let people come into your company and audit you and there's danger there, right? They might take with them like IP or they might find sensitive things and then like, you know, which were unrelated to the thing you wanted to get audited and all these bad things can happen, right? So there's like this potential, I think, for AI to give us a much stronger auditing system that can be part of how we build trust, how we let people build trust in us and the companies that, and products that we're building. So that kind of leads me to this next question, which is, should we be doing that?
Are there some other ways that you plan on building trust with your users? Or should we be doing this? Should we be making, like, not just public commitments. Companies make public commitments all the time. Like, oh, we care about this. We care about open source. We care about doing the right thing for the user. But are you actually willing to, like, make that a binding statement? Are you really willing to, like, stick your neck out and say, yeah, not only do I say that this is what I believe to be valuable, I'm willing to commit to an ongoing audit from some neutral arbiter, from some neutral AI-powered system that will come in and inspect every single thing that happened in my company, every Slack message, everything, and ensure that actually we are making decisions that abide by the mission statement of the company. That would be something that might have teeth.
Maybe this isn't the right way to do it, but I think things like this are going to be possible soon, but not possible today. And they might be the flavor of things that we need to start building trust once we're in this world where we've lost a lot of trust. Related question around alignment. So I think a lot about alignment. And I think there's this question of what parts of alignment do we have to solve? One reason we're working on alignment is because of the control issue. We want to make sure AIs stay under the control of humans. But I think there's this extremely...
high pressure question for the next 12 months, which is what parts of alignment do we have to solve just to make these models more economically viable, right? Just to make sure that the agents that all the startups are building as their horizons get longer and longer, that we can actually trust those agents are like not going off the rails, right? And if you use Cloud Code today and it works like five minutes at a time for you, that's like one thing because you're going to review a lot of what Cloud does. But if you're going to trust an LLM to work for a day at a time or a week at a time before you intervene, you better have some degree of certainty that it's not going completely off the rails, right? So I think this is like a huge open question. I'm actually really positive and bullish that there is this economic pressure in a good way to make progress on alignment because long horizon agents require it.
But I think it's an open question how much and what aspects of alignment have to be solved for this. Changing topics a little bit, I think this is a question that we all maybe think has been answered, which is, is there a set of data that can give you an advantage? If you go back in time just a handful of years before Frontier models, before LOMs, the assumption, not just the assumption, the fact was that custom data mattered. If you wanted to build an AI startup or if you're an enterprise and you're trying to deploy AI, you had this massive advantage if you had a massive data set that was custom to your need. And that was actually the only way to get useful AIs more than a handful of years ago was by training models on your custom data set.
And then very quickly, what we saw was LLMs got extremely powerful and extremely general. And actually, it was just better to use the general LLM than it was to train or even fine tune on your custom data, right? What I think is true is, maybe it's true, this is my open question, is there might be industries where that's not the case, where actually AI is maybe not great. An example might be like material science. Can an LLM do really good material science? Does a company that specializes in material science that has decades of data, can they do better? And I think potentially, yes, right? Like LLMs are great at everything that they've found on the internet, but are they great at all this like tacit knowledge that's locked up in companies that actually hasn't bled out? I think about like TSMC or ASML, right? Like they make these multi-billion dollar bets.
And they do an incredibly good job of keeping all that tacit knowledge they build in-house. It doesn't leak out. Frontier LMs do not know how to build a cutting-edge semiconductor fab. That's an important fact, actually. And I think if you're a startup thinking about where there's a defensible position, and I'll touch on this again in a few slides, I think that's an important question to keep in mind. Okay, different tactics here is you're probably aware of capacity issues. Everyone's trying to scale extremely quickly. I think there's demand from consumers, from your consumers, potentially from your startups, to scale extremely fast, right? Maybe we want to scale like 100x for the next couple years. That's faster than we can scale GPU production. So what are we going to do?
And I think there's a lot of open questions around, like, does fine-tuning actually matter? I think a lot of people have abandoned fine-tuning and said, actually, I'm just going to do better context management. Or is there a role to play in better routers between which model is small versus large, et cetera? So I think this is a place where, if you're interested in the technical details, you can have a competitive advantage, at least for the next year or two, because capacity is really going to matter, right? Often, from a product perspective, I like to say, make it great, then make it scale. If you're already starting to work on the make it scale part,
this is really important to you. And this can be some of the technical moat that you can build that gets you ahead of the competitors. Getting ahead of the competitors, though, is like, it's a rat race. So at some point, the models will get better, the capacity problems will improve, and then that advantage might go away. So you need to have a better answer to this question of, like, what's your moat? How are you going to stay ahead? What makes a durable advantage in a post-AGI world, right? In two years or three years, if I can just prompt, you know, Claude 7 or GPT-7 to just replicate your startup.
what's your advantage going to be, right? And if I'm a megacorp and I have more money than you and I can throw more tokens at that, like, are you really going to have a durable advantage or are you just going to get clobbered by megacorp? I think it's like a real question that is extremely serious. Even before AGI, I like to work on hard problems. Like, that's the moat for me. Everyone has a different type of moat. Some people are great at marketing, whatever, right? I like solving hard problems. But I think about what does it mean, like, to be a hard problem? Like, what's going to be hard in a post-AGI world? And I think a lot of things, actually, like TSMC, like ASML, those are hard problems that eventually will get easy.
with robotics, et cetera, but robotics are lagging behind. So I think there's these hard problems that will exist even in two years. And if you're willing to go after hard problems, like if you have the guts to do that, then you can have a massive competitive advantage, right? So like, what is that set of hard problems? You know, infrastructure and energy and manufacturing and chips. What else? Like, those are the things that are top of mind for me. But like, what are your answers to this question? Like, what's still gonna be hard and worth doing? I think about this question a lot too. Like, is there...
an intelligent ceiling to what you need for various different tasks, right? Like, you know, if you look at image gen, for example, you know, just recently with Veo3 with video gen, it's like, I can, I finally am getting fooled by videos. And like my Instagram feed is starting to get full up of like copybaras singing songs and shit. And I'm like, actually, this is great. I love these videos. That wasn't true three months ago, right? But can it get even better? Or is there like for a specific task for a specific use case, is there like some maps where it's like, yeah, that's, it's good enough. We've saturated, right? Like that's the best you can get at writing a poem. That's the best you can get at writing a get diff for a PR. That's, you know, an important question because if there is a ceiling,
then the commoditization for that task is going to hit much sooner, right? You actually can't stay on the edge longer by moving to the next model, figuring out how to prompt the next model. The task saturates, right? So the commoditization pressure will be even more extreme. So I think it's important to think about this. And again, I think it's going to depend on the task and the vertical. Some things, maybe there won't be a ceiling. Some things, maybe there will. This is changing tack a little bit here, but is there going to be a need for neutrality? Since the dawn of chatbots two or three years ago or whatever, people have been complaining about refusals as an example. Oh, I asked the model to do this, and it refused. If we end up relying on these models, that's a massive...
question for society right that there's going to be a handful of corporations that get to decide what is okay and not okay for an ai to do for you and if we start relying on ai to do everything then those companies become arbiters of what gets built right that's extremely important so is there is there going to need to be a notion of neutrality like we have today with some forms of infrastructure right electrical infrastructure is is neutral like if ge owned all the electrical grid they were like you can only use our electrical grid if you promise to plug in our toaster oven to it like no other toaster oven is allowed to work on our electrical grid that would be a problem um and it's good that didn't happen obviously we like fought and lost this battle for the for the web um what's gonna happen with ai Do we need AI neutrality, token neutrality? I don't know what we call it. I'll wrap up my set of questions here. I have an infinite number of questions, and I hope that these just stimulated you to think a little bit as well about...
not just these questions, but your own questions, right? I want to just like touch on one thing that's like always inspired me about Silicon Valley. And it's this like cringe thing that has always like, you'll only see it in ironic sense, but actually I think it's really great, which is like in Silicon Valley, we always used to say like, what's your startup doing? And someone's like, oh, I'm trying to change the world. Everyone always wanted to like think big and do the big thing. And like at the end of the day, it was a cat sharing app or whatever, right? But like the desire to change the world was there. I think startup founders really meant it when they said, I want to change the world. I think the thing that's concerning to me is, you know, I've been on the bleeding edge of AI for like ever and I've...
you know, been worried about AI risk forever. And, you know, five years ago when I talked to someone about AI, they'd be like, what the fuck are you talking about? You think these things are going to maybe kill us or whatever? Like, this is insane. Now when I talk to people, you know, I sit them down and I explain everything that's happening. They already are aware, obviously, but I'm like, let me give you some more details. And suddenly they're like, oh, you can see the gears start turning. And just regular folks that I talk to in my life, they get it. And they're like viscerally aware that the things that we're about to go through are extremely important. They're like humanity-defining, society-defining. Like people get this, right? It's not a nuanced thing, right? Where for the first time...
ever bringing a second intelligence in the world that matches humanity and will eventually exceed human intelligence. That's just obviously extremely important, right? But I get a lot of people to the end of that chain of thought. You can see the gears turning and then they ask a question and the question is, okay, how do we make money off of this? And I get extremely disappointed in that person. And I totally understand why they're asking this question, right? Like, you know, there's a lot of fear in the world. I think the fear of like all of us losing our jobs or not being able to build a startup that's going to be competitive anymore, or maybe the door to build a startup closes forever in the next few years. Who knows, right? Like there's a lot of fear. And I think AI is scary, right? So like people start thinking, well, I need to make money right now. Like if the economy is going to be fundamentally different in the next few years and I can't guarantee my place in it, like I better like make my mark right now. I better make my money right now. And I think that's totally understandable. So like, actually I'm like, yeah, okay, like here, let's brainstorm some ideas for how your next company can make money. Or if you're, you know.
investor or your employee whatever it is let's brainstorm your career so i'm always happy to help people with that but i also think that in that same moment, this is the last opportunity that we might have to make a difference, to change the world. If you're building a product, you need to use this moment, right? Like this might be the last product you build. It might be the last company you build. If you're inside of a company, the same applies. This might be the last chance that you have over the next couple of years to make that impact that could change the world, even in a small way. So if you have something you care about, now is the time to do it. I think about YC a lot and the slogan, first part is YC slogan, build something people want, right? That's what we all wanna do. And I think the truth is what people want often is something that's good for society, but you do need to think a little bit deeper.
Like I think people do want things they can trust. They want agents they can trust. They want bots they can trust. They want to know that when they use a product, it's not just going to like delight them for the next 20 seconds, that it's going to be good for their mental health for the next 20 years or the mental health of their children or their neighbors, right? Like we want good things. We as consumers and users, we want good things, right? And when we say build something people want, don't just think about what people will consume. Like what does society need? I think if you build the right thing, a lot of people will want it. So I'll wrap up and I'll say like, I know there's a lot, like I get stressed out thinking about this sometimes.
But I really think as founders or as people who are interested in being founders, we have this unique perspective. We think about things in a way that most other people don't. It's kind of the whole job of being a founder is finding an edge. That's the whole thing. And I think over the next couple of years, things are just going to move extremely fast. And the rules are going to change every six months. And you're going to have to think again. And if there's anyone that's positioned to stay at the bleeding edge of that, understand how those changes are impacting some of these questions and other questions that we're talking about, and then use the insights from the answers to those questions to drive positive change.
If there's anyone that can do that, it's you all here. It's folks that care about thinking about these things and care about building things that people want. So I hope that that's what you all do. I hope you make money doing it too while you can. And yeah, thanks for listening. I think we got time for some questions.
Thank you so much for the talk. I think that was probably one of the more grounded and down-to-earth talks I've heard, especially the last point about building something people want. I've always thought it's building something the world needs is perhaps a bit more important for a time like this. So I guess my question for you coming up with all those questions on how to approach AGI and the insights that come along with it, what are a couple sources of information that inspire you to do this, whether that's people, podcasts, books, and things like that? What would have been most helpful for building your mental model? Yeah, that's a great question. I hesitate to say it, but I think the honest answer is Twitter.
I'm really religious about curating my Twitter, though. If I see someone who I think has good takes, I follow them. If they have dumb takes, I unfollow them. I think you really need to be the master of your information diet. Not because I'm not worried about being influenced by bad opinions. It's more that I've just got a limited energy budget to digest new ideas, right? So don't just maximize for people you agree with. Maximize for diversity, right? I'm in reinforcement learning context all the time, and there's this notion of diversity in RL, like exploration versus exploitation. You want to make sure you're doing a lot of exploration in your information diet before you're doing the exploitation, which is like starting a company, for example.
Yeah, thanks for the amazing talk. It's probably my favorite talk today. Something I think about quite a bit is, like, in a world where AGI is coming, say, in two or three years, traditionally, like, the questions I ask myself is, if I have a couple of startup ideas, should I work on something that I'm really passionate about and have experience with expertise in? Should I work on someplace, an idea where the market's underserved or not as competitive? In a place where AGI is coming, should the most singular important question be, what idea is the most defensible against AGI? I mean, it's a great set of questions. I think even before all this AGI stuff, a lot of those were good questions. Like, personally, my opinion is that once you're, like, six months into 100-hour work weeks,
I don't care how passionate you are about an idea, you're going to fucking hate it. And the only thing that's going to keep you going is your desire to have the impact, your commitment to the company, your commitment to your co-founders and your team. So does it really matter to be extremely passionate about the domain that you're going after? In my opinion, no. But I think other people feel differently. So that's a personal question. But I do think being impact-oriented is really important. And whether or not you're trying to have an impact or not, I think defensibility really is, in my opinion, one of the key questions. I mean, that's kind of half the talk, I guess. It's like, what's going to change? And is the thing you're building just going to be a rounding error over the next six months? And I honestly think there's a lot of money to be made in the next...
on a six to 18 month horizon, if all you're optimizing for is hockey stick curve, grow your ARR, flip the company and make a quick buck, you might not need something long-term defensible. But if you want to build something that's going to stand the test of time and be part of this transition through the singularity and all the craziness of that, I would say think harder about the defensibility. That's probably the most important thing. Thank you so much. Yeah, good question. Some of those questions are really mind-bending. I've been thinking about them a lot as well. My question is, I know you've tried to keep it open, but what's your personal opinion on...
like the value of money do you think as the cost of goods and services go down money will become like less valuable because you know everything's free or it will become more valuable because we can do like way more stuff yeah it's a great question yeah i mean i think there's going to be policy decisions that impact us a lot right like do we need some form of ubi um or like you know maybe like slightly more weird like universal basic compute like if compute is the thing that powers everything are we all entitled to some form of compute these are like open policy questions that will start impacting that answer and i think we should think seriously about them because they will also change the nature of our society and the sort of like um checks and balances that we have over our democratic institutions right like if the government's giving out a ubi to everyone that's an extreme amount of power suddenly the government has over the entirety of society right but on the flip side if we don't do something like that i do think that we might end up in a world where You know, there's two primary forces at work today. It's capital and labor, right? You know, I won't say the names of people that I wouldn't want to work with, but there are people in this world that I wouldn't want to work with. A lot of people choose not to work for those people, right? That's like a massive hit to those people, right? But once AGI arrives, you don't need labor buy-in anymore, right? Like you don't need folks like me to approve of the thing that you're building or of your morals or whatever, right? Like capital begets capital in that world. And that can easily spiral out of control, right? We already have a lot of concentration of wealth. That can really spiral out of control. So I think we're like...
between a rock and a hard space, and I don't know what the answer is. But I think the policy decisions we make will have a huge impact on the question that you asked. Thank you. Yeah, good question. Hey, Jordan, thanks so much for the time. I guess the question I want to ask is, how do you think about alignment at the level of individual users? And I guess how important is that for trust, especially given that preferences evolve over time, and you don't want to keep retraining models, especially also when you have a lot of users? Yeah, I mean, I think everything, I have a startup product lens on everything, right? And it's like...
startup mantra is that users don't know what they want right and i think that's true to a large extent right but i think users still have values and you want to discover those values and be and honor them to some extent right so i think like something on top of mind for me is you know you probably saw like the sycophantic behavior from the recent you know other other ai provider and like i think if you put in front of a user two responses and one of them is more sycophantic one's like glazing i guess is what the kids say nowadays right uh then a lot of users will pick the sycophantic response right like in that moment they're like yeah of course like of course the question i'm asking is a great question thanks for recognizing it right but i think if you take a step back and you say hey hold on a second here's two principles and you can choose the ai that follows this principle or this principle and the first principle is that we're never going to blow smoke up your ass uh we're only going to tell you if we like something or if an idea is good if it really is uh and the other principle is actually we're just going to like you know we're just going to glaze you all day like that's what we're going to do if you ask the user which principle do they want almost everyone's going to say the first one right so i i think Doing what the user wants, you get to a different answer depending on what level of the engagement you're asking them. And I think that's really important because people can abuse that to their advantage by only asking the user the question in certain ways, right? And I think you need to really ask yourself, what's the right way to ask the user this question to get at the heart of what is going to be best for them? I don't know if I guess you kind of went off on a diet drive, but... That makes sense. Thanks so much. Cool. Great question. Yeah. Thank you for the talk. I really enjoyed it. It seems to me like you value critical thinking a lot. So I was wondering what topics or what like...
What opinions does the general crowd and the tech field have that you disagree with? I guess my high-level general statement that I'll tell you, and I'm happy to talk more offline, I think despite the fact that we say our industry is this forward-looking industry, and we're bold, and we love taking risks and putting everything on the line and seeing what other people don't see, I think the truth is that there's an extreme amount of groupthink. The products that we build, what gets funded by VCs, et cetera. And I think even today, you talk to a bunch of VCs, and they're like, oh, yeah, I'm ahead of the curve. I'm investing in AI. I'm like, no, you're two years behind already. Tell me about what you think needs to happen.
You mentioned how trust is going to be a bigger issue in the future. So do you think that blockchain might be a part of the solution there? Thank you. I will just preface by saying that I'm a huge blockchain doubter. I don't know. But nonetheless, the price keeps going up, and I don't get any of that because I refuse to buy it. But I do think in this world where we need trust, yeah, that's the right set of ideas. We need ideas like that in others, right? Something that I was touching on is AI-powered audits. Can two different AI companies audit each other? How do we get to places where we can build trust?
If we do end up with some form of universal basic income or basic compute, basic tokens, does that need to be mediated by a blockchain so that it's not just at the behest of a central government? Yeah, I think those are reasonable questions that maybe I could get behind a blockchain on. Good question, yeah. Thank you. So Google recently released an ATA protocol to standardize how agents would talk to agents. And we talked a little bit about trust on agents. So I'm curious how you would envision a world of agents talking to agents and how would that affect applications and would there be an agentic premise scheme? Yeah, it's a great question. I could talk about this all day and I'm almost out of time. So maybe I'll give you one example where it's not obvious.
why this was hard but like think about like a personal assistant agent that's just scheduling meetings for you right seems trivial right like oh yeah i'm just gonna look at this person's calendar and then i'm gonna like suggest times right but actually like you've already lost the game of being a good personal assistant because there's a game theory component to how you schedule meetings right and if you're like too liberal with like showing that there's free slots you're like communicating this person's not busy right uh or you're communicating that like it's really important to have this meeting with the person that you're scheduling uh whereas like if you say yeah like i'm happy to schedule a meeting between you and joe but it's like two weeks out three weeks out right like there's a power dynamic that you're you're doing right like the truth is like all these game theoretic things matter a good assist a good human assistant knows all these things right and like them but it's all implicit right it's not like oh there's this like concrete piece of information that the agent has access to and that's the important part of the security it's like way more subtle and semantic so i think it's just hard but i think it's a great question yeah thank you cool i think uh out of time unfortunately but feel free to like shoot me a message online or i don't know how to get like my twitter's jordan fisher jordan fisher shoot me a message always happy to chat and thanks for the great questions and hope you all enjoyed the talk