AI is moving fast across the enterprise, but without visibility, it's just chaos. ServiceNow turns that chaos into control. With the AI control tower, you see all your AI in one place. To put AI to work for people, visit ServiceNow.com.
I'm Mr. Klein, and this is the Ezra Klein Show. Over the past, I don't know, five, six years, I've been very influenced by the affected Altra is a movement. On one level, effective Altra is a bit simple.
It asks, how do we do the most good we can with the money and the resources we have? That turns out to be one, a deceptively difficult question, and two, weirdly, one that we don't ask all that often. One that oftentimes you think people are asking and they are not. But the difficult parts are maybe more interesting.
How do you measure the most good? What about when you think something is good, but it cannot really be measured? Who defines good? Who verifies impact?
How do you judge the value of supporting art against the value of building housing for the poor? Effective Altra has roots in the academy. Philosophers like Toby Ord and Wilma Cascol, Peter Singer, they've been central in creating the movement. And importantly, they're central in the way the movement thinks and reasons.
The culture of effective Altra is in my experience. And this is both its best and worst quality in a way. Can feel like a philosophy grad seminar that never ends. By that I mean it delights in taking the logic of its questions as far as it will go.
It's unafraid, even ecstatic to follow answers that strike others as very strange or unintuitive, sometimes even cruel. It's always questioning its own assumptions and everyone else's. It can, in my view, sometimes be performatively cold or logical in a way that's actually quite narrow about human flourishing. But as I've said, I've learned a lot from these thinkers.
What's interesting to me is that effective Altra is tend to come out on two very different ends of what you might end up worrying about. Some take the analysis and end up worrying a lot about very provable, very, very well defined interventions like distributing malarial bed nets because those are the interventions we have studied with the best research we've done randomized controlled trials and we know they save lives cheaply. And then others come out obsessed with much more speculative threats like say killer artificial intelligence because if you start running the thought experiments, if you assign a moral way to the future and all the potential human beings in the future, anything that can end humanity tomorrow really demands our attention today. If you can improve the likelihood of countless generations, even a little bit, that really nets out to a big impact on the future.
No one represents the dual and sometimes warring impulses of effective altruism better to me than holding Krenowski. Holding was one of the co-founders of Give Well, which measures the effectiveness of different charities and recommends the ones his most confident can save lives cheaply. But then he spun out of that to found open philanthropy, which is more on the speculative side of effective altruism, the thought experiment side. But because he's actually giving away money and making grants and shaping sectors and commission research, he and his team have to be pretty serious about how they approach questions that are not always considered seriously.
They don't even always sound serious when they are spoken aloud. So there is a practicality, even a groundedness to his analysis, even when it ends up in very, very strange places. And more recently, Holden has been going to stranger places still. He launched a blog called Coldtakes and on it, he's making the case in piece after piece that we live in maybe the most important century humanity will ever have the most important we have had or will have that the future could be wildly unlike the past.
It is real mind expanding stuff. And this episode warning will go to some very strange places. This is not our normal fare. But I urge you to follow it there.
There's a concept Holden and I talk about here, worldview diversification, the practice of recognizing that we're not always sure how we should look at the present or the future. And we should keep a number of different possibilities, some of them even out landish, alive in our minds. We should even keep them alive in our actions, rather than trying as we so often knew to choose between them. This is something I tried to do on the show.
You can think of each episode as a little exercise in worldview diversification. And I try, as you know, to stay away from the choosing, except when we really have to. But it's particularly something we do in this episode is hold it right about some of the more wild predictions we discussed at the end. I don't know.
On some level, I've a lot of trouble believing that. But I do think it's good to stretch the predictions you're willing to entertain, if only to help you see the present more clearly. As always, my email at mytimes.com. Hold on, Krenowski.
Welcome to the show. Thanks for having me. I've wanted to do this a long time, man. Cool.
It's a minute in the making. I want to begin at the beginning for you before cold takes, before open fill back in the in the gable days. Tell me the story of why you started givial. Sure.
I was a few years out of college and I wanted to give to charity. And I had the immediate thought that maybe I could find a website that would just tell me where to give to charity to sort of get the best deal. I think you could maybe think of what I wanted as a wire cutter for charity, so there wasn't a wire cutter then either. What I wanted was to kind of help as many people as I could with the money that I was giving.
And I tried to find this and I couldn't. And my coworker, Ellie Hasenfeld and I were in the same boat on this. We were going through the same journey together. And we both decided to create the website that we wish existed, which is Give Well, which is just public recommendations telling people which charities can help their money help the most people possible.
So I looked for things like this in this period. And I found things that at least purported to be this. People might be familiar with charity navigator. There's a lot of cross-charity budget comparisons.
Why weren't those what you were looking for? Sure. So at the time we were doing this was around 2006, 2007. And the obsession in the charity evaluation world was around the overhead ratio or how much of a charity's budget does it spend on the so-called programs versus the so-called overhead.
This is an appealing thing in that you have to report these numbers to the IRS so you can get this number for any charity. But it really doesn't seem all that relevant to me. For example, if you have a charity and you decide to pay your top talent more or you get better information technology and now you're doing way more good and helping way more people, is that overhead? Is that somehow a waste of money?
Is that somehow you did the wrong thing? And so we wanted something different. What we wanted was this question of what charity is going to help the most people per dollar that I spend. And the crucial difference is that we were not in a place where we had a charity we already liked and we wanted to look it up and make sure it was legit or that it wasn't fraudulent.
We wanted something different. We wanted recommendations of which charities out of all the many you could do the most good. So kind of like how the wire cutter will actually kind of tell you what five products to maybe buy instead of you look up a product and they tell you if it's going to break. I want to know what the wire cutter is now in New York Times company and I've not paid hold into make all these wire cutter analogies.
But getting at this, one of the reasons there was an interest in things like overhead spending is it is cross comparable across charities. As you mentioned that that's a number reported to the IRS. There are a lot of charities that do very different things. Some of them work on international poverty, some of them work on public health, some of them give money to ballet.
How do you begin to narrow down and test which ones do the most good for the least money? Yeah. So we started with some criteria. We wanted to find charities that were proven, cost effective and scalable.
Proven means that there's strong evidence that what they're doing helps people. It's effective means that they're helping a lot of people per dollar and scalable means that there's room for more funding. It means that if you give more money, more people will be helped. So we're not just talking about a charity that did something great once and wants more money now.
Those criteria will not capture everything good and I imagine we'll get to this that I've later branched out in other kinds of giving that don't have necessarily all those criteria. But when you have those criteria, if you look at any charity and you ask whether it's really helping people, you've got an academic literature review. Does education interventions, do those help children learn better? Do health interventions help people live longer?
And what you can do is you can start looking for the programs that are the most proven and the most cost effective and then you can find charities that do them. So that's the way to narrow the field. And one of the things that we learned early in GiveWell, which is a lesson that's kind of stuck with me throughout my journey to where I am now, is that if you want to do the most good, in some ways the worst place you can start is your neighborhood, your friends, your country, because if you look at the whole population, in this case with GiveWell, there's just a lot of people living in countries that are extremely poor by US standards and your money can go a lot further there. You can help a lot more people with the same money if you're willing to broaden your moral circle and include more people in it.
Talk me through this tangibly. What is the first charity you find where there is really, really good evidence that it can help people for very little money? Where do you find the charity and where do you find the evidence? I could talk about the literal first we ever recommended, but instead I'll talk about the first where it really started to come together that we felt we were nailing these criteria.
So that was the Against Malaria Foundation. And what they do is they basically distribute insecticide-treated bed nets to help reduce the burden of malaria. Now, insecticide-treated bed nets cost about $5. They often cover two people.
Sometimes they cover one person. They last for several years and they can kill mosquitoes and block mosquitoes so that people don't get malaria. There's a large number of very rigorous, randomized controlled studies on bed nets. And if you look at the effect size, it looks like they are helping people very cost-effectively in the sense that there's a death averted for every few thousand dollars that you can spend distributing bed nets, which is kind of incredible and has been very hard to beat that number at any point during Give Well's Journey.
So that's the intervention that's distributing bed nets. And then what we found is we looked at several organizations that distributed insecticide-treated bed nets. And one of them was really tracking the whole process. They were providing documentation that the bed nets were actually handed out.
They were providing shipping information of the bed nets arriving. And over time, they would add more data on going back and surveying people if they were still using the bed nets. So now you can put together this whole case. We have a very strong evidence base.
You have an amazing deal in some sense where a lot of people get help for each dollar you spend. It's very hard to do that much good, helping people in the US with donations. And then you have this organization that is really repeatedly carrying it out and that is providing all the information we need. And so it becomes this very exciting way to spend your money.
So I want to hold on this idea that giving two anti-malaria charities could save a life for a few thousand dollars or specifically according to the Give Well website three to five thousand dollars. How does that compare to more conventional causes that people may be familiar with like disaster relief or donating to a soup kitchen? It's about the best we've been able to find in some sense. We've looked high and low for ways to help people, for small amounts of money.
Some listeners might be a little confused when they hear three to five thousand dollars because they might have heard, you know, well, I saw a charity that could save a life for 20 dollars. But at Give Well, it's always been about rigorously investigating the numbers and subjecting them to all kinds of scrutiny and analysis. And when you really go hard on the analysis and you ask that the number be real, you know, 20 cents to save a life is not something that I think you really have the opportunity to do with your giving and three to five thousand dollars. It's just empirically.
It's been, I mean, I think that's just incredible compared to most things that are out there. What are some of the other numbers you've come to when you've looked at other methods? A lot of times we don't put a number on it so much as we just put a bound on it where we'll say, hey, the evidence here is not very strong. The intervention itself is very expensive.
It might be like thousands of dollars just to like put a person in a program that might or might not be helping them at all. And so a lot of times it's more a matter of saying this doesn't have a strong enough case or it's just intuitively too expensive. It's not going to match that other figure. So one of the critiques of Give Well, Give Well model is that to use the old line about economics, it only looks under the light, right?
It looks for your, it looks for its keys, but only where the lamppost is because a lot of things that could be good in the world and they don't have a very, it is hard to run a randomized controlled trial of them because they only have in once or they are affecting something more diffuse. So what was the thinking behind demanding is very high level of parable proof, which on the one hand, like you say, here's where my dollar is going, but on the other hand, you might say, yeah, but the leverage on that dollar is smaller because I'm only trying to save the one life as opposed to influence something to save 10,000 or a million by earning a war or changing civil service or whatever it be. Right. Right.
I have a bunch of thoughts on this. The original thinking was just very pragmatic. It's just a startup saying, let's do what we can do. I think sometimes you do want to start under the light.
If there's a big area under the light, well, if your keys happen to be there, you're going to find them a lot more quickly. Maybe you should start there. So I think in some ways it's not always such a, such a bad thing to do. You know, another intuition we had is just that a lot of things that people try to do to help people are just very speculative.
They're often based on having certain feelings about the world or feeling like certain things kind of just feel supportive to do. And we thought that things that are really supported by evidence where you can really drill down and see how much money's going there might actually be systematically better because they're kind of, they're optimized in a different way. Another way you could think of this is the less you know about some intervention, the more you might expect that it's going to just be the average thing you can do. And when you can create a strong evidential case that your money's doing credible things, that might actually be better than other options.
The final thing I'll say though, is that I at least have branched out a lot since then. So I now run a different organization or a COCO run a different organization called Open FADY that does not have the same requirement that everything be totally proven based on evidence. And an interesting thing is that we've been looking for things that are better than Give Well, Top Charities. And it's been really hard.
It's really been surprisingly hard. Yeah. I'm going to hold you there because we're moving to OpenFilled. But I want to do this a little bit more slowly.
Sure. You're running Give Well with Ellie. It's going well. Give Well became a big player on the block right quickly.
I've given a lot of money through Give Well over the past, you know, five, ten years. What makes you decide to split off and start a new organization? Well, we met Carrie Tuna and Dustin Moskovitz. Dustin is one of the co-founders of Facebook and also Asana.
And they were trying to give away their fortune in a way that would help the most people possible. And we just felt that a different approach might be called for when you have a public website making recommendations to anyone and everyone versus working with one family that's giving away a huge amount of money. The second one kind of starts to put new options on the table. And so OpenFilled to me still recommends a lot of donations to Give Well, Top Charities.
And Carrie and Dustin still give a lot of money there. But there have been other things that may be higher risk or maybe more in the mode of what I call hit space giving, which is that if you can get the occasional success that is a really huge win, that might make up for a lot of donations that don't quite have the effects they wanted to. And so that's what that's what caused us to kind of pivot. And we started OpenFilled to me as a project within Give Well and it eventually spun out.
Before you begin OpenFilled, you undertake this big study of the history of the philanthropy space. Tell me a bit about that study and what you learned from it. When we started working with Carrie and Dustin, I wanted to understand a little bit more about how Big Philanthropies in the past, what they had accomplished. And I didn't know what I'd find it.
I wouldn't have been surprised if I learned that actually they'd never accomplished anything and we should go back to doing the simplest stuff we can because it's that hard to help people. But that's not what I learned. What I learned is that there have been incredibly impressive successes from philanthropy that I think rank up there with the most important events for human welfare in the last hundred years or so. We actually at OpenFilled, we've now named about half our conference rooms, each one is named after a Philanthropic Success story.
So what are the names? One of my favorites is called Green Revolution. And that was when the Rockefeller Foundation funded Norman Borlaug and others to research improving crop yields in Mexico. If I'd been around at the time, I doubt I would have said, oh, improving crop yields in Mexico.
That's going to be the biggest success that's ever had by philanthropy. But it may have been because this is generally now credited with kicking off the Green Revolution where a bunch of countries went from importing to exporting food and it's credited with saving a billion people from starvation. Norman Borlaug ended up winning the Nobel Peace Prize because it turns out that this was a very scalable improvement in agricultural productivity. So once they had these crops, they could just breed them anywhere.
Agricultural productivity took off in a lot of poor countries and that kickstarted all this economic growth. Another one of my favorites that I'll just throw in is the pill, the common oral contraceptive for birth control, the work was funded by a feminist philanthropist named Catherine McCormick. And at the time, this is the kind of thing that wasn't going to get funded by the government because it was controversial. And in fact, they weren't able to advertise the pills birth control originally.
Instead the warning label was the advertisement. They had to put on a warning label that it could prevent pregnancy. So as an example of philanthropy being ahead of the curve, doing something that was controversial, but they ended up with something that was transformative and was a huge moment for feminism and human welfare because they're willing to be a little controversial like that. One thing I noticed about both of the case studies you use there as examples of huge wins is their technologies.
The Green Revolution or new kinds of crops, the pill is a medication. I don't think most philanthropy is about seeding and staking new technological development. I'm not saying none is. I'm back up a friend who's working on that kind of thing right now, but not most of it.
So is that a problem? Is that one of the places where open pill begins to diverge? If you should actually be doing product development through philanthropy? Phalanthropy is not something I would say is neglected by philanthropy or probably right that most philanthropy is not technology philanthropy.
There is a lot of philanthropy in science, especially today. It's become a very fashionable thing to put money into. But I would say it does reflect this idea that if you want to do really incredible things that have massive scale and help tons of people having innovation, having new technologies that can be copied and used freely by anyone is a great way to get leverage. It's a great way to just have really big effects.
And if you don't have any theory in your philanthropy of how you're getting massive leverage and affecting huge numbers of people, then you may be better off with the most straightforward cost-effective proven stuff. Well, let's talk about how to put that kind of theory into practice. Give me some examples of what you end up funding from hit space giving. I mean, one example is just where the largest funder in the world of farm animal welfare, that's both attempts to develop alternative foods that can reduce meat consumption, but also corporate campaigns to try and put pressure for better treatment of animals.
Over the last few years, basically every major grocery and fast food company in the US has pledged to go cage free. We've been funding a lot of the work that led to that. It was already going we came, but we tried to speed it up. And we've also taken that work global and been funding a lot of corporate campaigns globally.
And so while we are a fairly large philanthropy, we're not the largest, but we're the largest funder of that work because that's one of these things where I think we might look back hundreds of years from now and say, that was the greatest moral issue of our time. That was this unacceptable treatment of these creatures. We've now decided our hindering creatures that we should care about. But at this time, this is just not an issue that really tends to fire many people up.
And so we're doing something others won't do. And I believe there's been a lot of impact and there's been a lot of difference made because we're willing to go into kind of a weird cause. Some other examples, I think we were the first major institutional funder of the Yimbi movement. This is the attempt to advocate for less restrictions on building housing to make housing more affordable.
And this is again, just something that was kind of new and weird and now is now a combination wide movement. And that was not how it was when we started funding it. We funded macroeconomic stabilization policy. So this is a bit of a wonky one.
How does the Federal Reserve prioritize full employment versus controlling inflation? We believe that this is one of the most important things in the whole world for the welfare and bargaining power of the working class. And yet it's an issue that people often ignore. They think of it as a technocratic issue, whatever the experts at the Federal Reserve will decide what to do.
They don't see it as an area for philanthropy. And I think we're one of the only philanthropists in that area when we came in and still now. But I think it's tremendously important. And we funded a lot of analysis and even advocacy on how to trade off full employment and controlling inflation.
A final example is we've had a biosecurity and pandemic preparedness program since about 2015. I'm certainly not going to say I've been happy with the preparation response for COVID. But I think it could have been worse. And I think the organizations that have played important roles, you know, by the time we all knew about COVID, it was too late to come in and support them for years and help them be in a solid position and build up a deep bench and a lot of expertise.
And it was back when that was kind of an unusual cause for philanthropy to be in that we were supporting all that work. Let me ask about a couple of these. So macroeconomic stabilization is an interesting one. Because one way of asking that question is why do you think you all understand macroeconomic stabilization better than the Fed and others?
You're a young organization, you're a young guy, you're not an economist, nor most of the people work for you. A lot of people worked on this for a long time. You're coming in saying there's a huge untapped opportunity here. I would understand for a very explicitly political organization to come in and say, oh, we think the Fed has gotten it wrong.
We want more full employment. But what is the difference you have convinced yourself you can make on it? Well, it's important to understand the general structure of open philanthropy is that we consider our expertise in finding causes that are important, neglected, and tractable. Tractable means there's something for us to do.
And so we try and find the right problems to work on. That's what we consider our comparative advantage. And then when we are doing work, we are hiring and we are funding experts. And so this is not about holding and going and learning all about macroeconomic policy and then going and explaining to the Federal Reserve that they got it wrong, that's not what happened.
We funded groups that have their own expertise that are part of the debate going on, their experts on both sides. But we funded a particular set of values that says full employment is very important if you kind of value all people equally and you care a lot about how the working class is doing and what their bargaining power is. And historically, the Federal Reserve has often had a bit of an obsession with controlling inflation that may be very related to their professional incentives. And so we do have a point of view on when there's a debate among experts, which ones are taking the position they're taking because that's what you would do if you were valuing everyone and trying to help everyone the most versus which are taking position for some other reasons.
So we didn't roll our own macroeconomic policy insights. We funded experts, we funded think tanks, but we do have a point of view on what kind of value you should be driving that expertise. I think something striking about that list is the sheer diversity of things you all fund, not only in terms of causes but categories of causes. And this gets to what I think of as one of the most interesting things Open Philanthropy does, which is the way you intentionally divide up your giving portfolio into buckets based on really different ethical, arguably even metaphysical assumptions.
So tell me about worldview diversification. I need to start with the broader debate that world of diversification is a part of at Open Philanthropy, we like to consider very hardcore theoretical arguments, try to pull the insight from them and then do our compromising after that. And so there is a case to be made that if you're trying to do something to help people and you're choosing between different things you might spend money on to help people, you need to be able to give a consistent conversion ratio between any two things. So let's say you might spend money distributing bed nets to fight malaria, you might spend money getting children treated for intestinal parasites.
And you might think that the bed nets are twice as valuable as the dewormings or you might think they're five times as valuable or half as valuable or one fifth or 100 times as valuable or one hundred. But there has to be some consistent number for valuing the two. And there is an argument that if you're not doing it that way, it's kind of a tell that you're being a feel good donor, that you're making yourself feel good by doing a little bit of everything instead of focusing on others, on being other centered, focusing on the impact of your actions on others, which you can get from there to an argument that you should have these consistent ratios. So with that backdrop in mind, we're sitting here trying to spend money to do as much good as possible.
And someone will come to us with an argument that says, hey, there are so many animals being horribly mistreated on factory farms and you can help them so cheaply that even if you value animals at 1% as valuable as humans to help, that implies you should put all your money into helping animals. On the other hand, if you value them less than that, let's say you value them a million as much, you should put none of your money into helping animals and just completely ignore what's going on factory farms, even though a small amount of your budget could be transformative. So that's a weird state to be in. And then there's an argument that goes, but even more than that, and this idea is called long-termism, if you can do things that can help all of the future generations, for example, by reducing the odds that humanity goes extinct, then you're helping even more people.
And that could be some ridiculous comic number that, you know, trillion, trillion, trillion, trillion lives or something like that. And it leaves you in this really weird conundrum where you're kind of choosing between all in on one thing and all in on another thing. And open philanthropy just doesn't want to be the kind of organization that does that, that lands there. And so we divide our giving into different buckets and each bucket will kind of take a different worldview or will act on a different ethical framework.
So there's a bucket of money that is kind of deliberately acting as though it takes the farm animal point really seriously, as though it believes what a lot of animal advocates believe, which is that we'll look back someday and say, this was a huge moral error. We should have cared much more about animals than we do, suffering is suffering. And this whole way we treat this enormous amount of animals on factory farms is an enormously bigger deal than anyone today's acting like it is. And then there'll be another bucket of money that says, animals, that's not what we're doing.
We're trying to help humans. And so you have these two buckets of money that have like different philosophies and are following it down different paths. And that just stops us from being the kind of organization that is stuck with one framework, stuck with one kind of activity. Before we move on, I want to pack this a little bit more.
So let's focus in on animals for a minute. You alluded to the fact that even if you assign a very low moral worth to animals or to their suffering, 1% or 0.1% of that of a human, that it ends up adding up to quite a lot. Can you run through that math for me and its implications? Well, the math would be that I mentioned before that if you're distributing insecticide-treated bed nets, you might avert the death of someone from malaria for a few thousand dollars, which is pretty amazing.
And it's going to be very hard to find better than that when you're funding charities that help humans. However, with the farm animal work, for example, the cage free pledges, we kind of estimated that you're getting several chickens out of a cage for their entire lives for every $1 that you spend. And so this is not an exact equivalent, but if you start to try to put numbers side by side, you do get to this point where you say, yeah, if you value a chicken 1% as much as a human, you really are doing a lot more good by funding these corporate campaigns than even by funding the bed nets. And that's better than most things you can do to help humans.
Well, then the question is, OK, but do I value chickens 1% as much as humans? 0.1%? 0.01%. How do you know that?
And one answer is we don't. We have absolutely no idea. The entire question of what is it that we're going to think 100,000 years from now about how we should have been treating chickens in this time. That's just a hard thing to know.
I sometimes call this the problem of applied ethics, where I'm just I'm sitting here trying to decide how to spend money or how to spend scarce resources. And if I follow the moral norms of my time, based on history, it looks like a really good chance that future people will look back on me as a moral monster. But one way of thinking just to come back to the chickens question, one way of thinking about it is just to say, well, if we have no idea, maybe there's a decent chance that we'll actually decide we had this all wrong and we should care about chickens just as much as humans. Or maybe we should care about them more because humans have more psychological defense mechanisms for dealing with pain.
We may have slower internal clocks a minute to us might feel like several minutes to a chicken. So if you have no idea where things are going, then you may want to account for that uncertainty and you may want to head your bets and say, if we have a chance to help absurd numbers of chickens, maybe we will look back and say, actually, that was an incredibly important thing to be doing. I want to note something here because I think it's both an important point, substantively, but also in what you do. So I'm vegan.
I've not, except for some lab grown chicken meat. I've not eaten chicken in 10, 15 years now, quite a long time. And yet even I sit here when you're saying, should we value a chicken 1% as much? You're not like, I don't like that.
To your point about what our ethical frameworks at the time do and that possibly a real comparative advantage is being willing to consider things that we are taught even to feel a little bit or pulse considering. How do you think about those moments? Like, how do you think about the backlash that can come? How do you think about when maybe the mores of a time have something to tell you within them that you shouldn't be worrying about chicken when there's many people starving across the world?
How do you think about that? That's a question. I think it's a tough balancing act because on one hand, I believe there are approaches to ethics that do have a decent chance of getting you a more principled answer that's more likely to hold up a long time from now. But at the same time, I agree with you that even though following the norms of your time is certainly not a safe thing to do and has led to a lot of horrible things in the past, I'm definitely nervous to do things that are too out of line with what the rest of the world is doing and thinking.
And so we compromise and that comes back to the idea of world of ve ve ve ve ve ve vacation. So I think if open philanthropy were to declare, here's the value on chickens versus humans and therefore all the money's going to farm animal welfare. I would not like that. That would be uncomfortable and we haven't done that.
And on the other hand, let's say you can spend 10% of your budget and be the largest funder, a farm animal welfare in the world and be completely transformative. And in that world where we look back, that potential hypothetical future world where we look back and said, gosh, we had this all wrong. We should have really cared about chickens. You were the biggest funder.
Are you going to leave that opportunity on the table? And that's a world of v diversification comes in where it says, we should take opportunities to do enormous amounts of good according to a plausible ethical framework. And that's not the same thing as being a fanatic and saying, I figured it out loud. I've done the math.
I know it's up because that's not something I think. I'm struck by that. I really like world view diversification as a way of thinking about things. And I think it's also relevant as an individual practice.
Something I see in my travels around the world of the internet is people are very intent. Even if they would not say they're 100% confident in their worldview, their political ideology, their whatever, they are really interested in making it dominant against all commerce. So just tell me a bit about organizationally, intellectually, the discipline of maintaining a certain level of agnosticism between worldviews, whose differences you can't really answer. So one of my obsessions is apply to epistemology, which is like just having good systems for figuring out what your beliefs are and kind of an overwhelming flow of information that is today's world.
And I think one of the tools that some people use for it that I find really powerful and I'm going to write about is what I call the Bayesian mindset, which is this idea that when you're uncertain about something, you can always portray your uncertainty as a number. And you can portray it as a probability. There's thought experiments, there's tools for doing this. You can say instead of something that's true or false, that it's 30% and you can look back later and you can see if things that you said were 30% likely come through 30% of the time.
I think this is a very powerful framework and using it can often get you out of the headspace of believing that things are true or false and just having degrees of belief in everything and often taking something very seriously, even when you think it probably won't happen, just because it's important enough and has a high enough probability that it deserves your attention. And on the other hand, I think this framework sometimes can take people back into a state of fanaticism where you might say, hey, here's something that would be a really huge deal and it's at least 1% likely. So that means it should be the only thing I think about. It should be my obsession.
It's like the examples I was giving before. And that, I think, just lands you in a similarly dogmatic place. And so OpenFlanthroby is kind of operating two levels of uncertainty. It's often using the Bayesian mindset.
But when the Bayesian mindset brings you to this implication that you'll have to be all in on one thing or another, we'll say no to that too. And then we'll just go to another level of diversification and we'll have different buckets with different philosophies on the world. I want to pick up on the fanaticism component. And I'm not accusing anybody of fanaticism, but one of my critiques of the effect of Algis world is that it can get very obsessed by that conversion number you were talking about a minute ago.
And it particularly I think is a culture as it has matured a bit more. There's now a aesthetic sometimes of being willing to take the most hard-hearted logic experiment seriously and show that you're the real effect of Algis because even though it sounds like a terrible thing to do, you ran the math and it's not. And well, but this is Wilma Cascale who's a philosopher and was a founder of the effect of Algis movement. You used to have this thought experiment where there's a building on fire and there's a family in one room who could die.
And then there's another room or I think it was actually an attached garage or something that has a bunch of very expensive art in it. What do you say? And the point of the experiment, originally, was you should, of course, say the family. And he was making the meta point that many people are donating to museums instead of to malarial bed nets.
I think now a lot of effective altruists would answer the other way because the point is well if that art is worth $500,000 and you can turn that $500,000 into X number of malarial bed nets that saves more than five lives. And so, of course, you need to do that. And I think that gets you into pretty dangerous territory. But I'm curious how you think about those questions.
I do agree that there can be this vibe coming out of when you read stuff in the effective altruist circles that kind of feels like it's doing this. It kind of feels like it's trying to be as weird as possible. It's being completely hardcore, uncompromising, wanting to use one consistent ethical framework wherever the heck it takes you. You know, that's not really something I believe in.
It's not something that open philanthropy or most of the people that I interact with as effective altruists tend to believe in. And so, what I believe in doing what I like to do is to really deeply understand theoretical frameworks that can often insight, that can open my mind, that I think give me the best shot I'm ever going to have at being ahead of the curve on ethics at being someone whose decisions look good in hindsight instead of just following the norms of my time, which might look horrible in monstrous in hindsight. But I have limits to everything. Most of the people I know have limits to everything.
And I do think that is how effective altruists usually behave in practice and certainly how I think they should. What do you think the limit of that actual law experiment is, of the just convert lives into money, you can save X number of lives for X number of money. And so if you get more money by getting the money, it's supposed to save the life, you should do it. I think there's a lot of problems with that argument.
And I could sort of go into them. So there's things about setting norms. There's things about following rules so that you don't want to be the kind of person who is constantly behaving in strange, unexpected ways and screwing over people around you because you've got this strange mathematical framework that's going on. So I think there's a bunch of things that are wrong with running and saving the painting.
But I think I also just want to endorse the metaphor principle of just saying it's okay to have a limit. It's a reflective equilibrium game. So what I try to do is I try to entertain these rigorous philosophical frameworks. And sometimes it leads to me really changing my mind about something, you know, by really reflecting on, hey, if I did have to have a number on caring about animals versus caring about humans, what would it be?
And just like thinking about that, I've just kind of come around to thinking, you know, I don't know what the number is, but I know that the way animals are treated on factory farms is just inexcusable. And it's just brought my attention to that. So I land on a lot of things that I end up being glad I thought about. And I think it helps widen my thinking open my mind, make me more able to have unconventional thoughts, but it's also okay to just draw a line.
I think it's okay to look at this art thing and say, that's too much. I'm not convinced I'm not going there. And that's something I do every day. AI was supposed to take work off your plate.
Instead, I gave you homework. ServiceNow's AI specialists do to work, start to finish so people can do the parts of the job their best at. To put AI to work for people, visit ServiceNow.com. Hi, I'm Solana Pine.
I'm the director of video at The New York Times. For years, my team has made videos that bring you closer to big news moments, videos by Times journalists that have the expertise to help you understand what's going on. Now we're bringing those videos to you in the Watch tab in the New York Times app. It's a dedicated video feed where you know you can trust what you're seeing.
All the videos there are free for anyone to watch. You don't have to be a subscriber. Download The New York Times app to start watching. We've been talking a lot here about how to value animals, but the other big worldview here is long-termism, which has to do with valuing future human lives.
So tell me more about that worldview. So the basic idea of that worldview is that if you can do something today that affects all of the future generations, then you have helped a truly mind-numbing number of people. We don't know how many people. We don't know how many people will live in the future, but it could be an extremely large number.
Most things that we can do today are not the kind of thing that we have any reason to believe will still matter a billion years or now, but some of them could be. Climate change could be. Climate change is something that could really, in theory, could imaginably knock humanity off course forever and causing it to be less likely that this happens could be the kind of thing that matters for every future generation. And so long-termism says there are all these people who don't have any voice today in the actions we're taking that affect them.
And so why don't we take the actions that will still matter a billion years from now because I've affected that many people and maybe that's the way to do the most good. So one of the critiques of long-termism is it quickly gets you into this kind of mathematical moral blackmail where, well, if you say, because in the future human beings could spread throughout the galaxies and there could be a trillion of us over and over again, there could be a trillion of us. So if you give the future human lives 1% of the weight of current human life or 0.1% sort of anything that makes that future more likely to happen is just an astonishingly good investment that outweighs anything you do for people today. How do you think about that?
I have a few ways of thinking about this. One way comes back to world-dudeversification again. So what we aren't trying to do is find the one master framework that is the one thing what we are trying to do is find things we can do that may turn out to be hugely ahead of the curve, that may turn out to be a really big deal, that may turn out to be the best money we've ever spent. I think long-termism definitely checks that box.
And so open philanthropy is never going to be an organization that is exclusively long-termist, but we do put a lot of money into it. And the way you phrase it is one way phrasing it. But if we also just phrase it another way and we say, why don't we try to focus our actions and our efforts on the things that might still matter a billion years from today? Why don't we try to do the things that optimize for having the best future we can, for the best thing we can to the future generations, for having the best overall story of humanity that we possibly can, having a healthy society, a society that makes good decisions, a society that's equitable and inclusive.
I don't know. I don't think that sounds really so crazy. And I think if you were to walk around all day, this is something I've increasingly been trying to do myself because we've been doing division of labor at Open Philanthropy and I've been focusing more on long-termism. But if you just walk around all day just thinking, well, what is it that's the best for helping the world be a good place a billion years from now?
I think you end up doing a lot of really super reasonable things and you're paying a lot of attention to things that should be getting more attention today. Well, talking through how you can think of what the long-term and long-termism is because the one might have of just the way human beings are right now, I don't think we're typically doing things to make 24 hours from now. The best thing could possibly be, right? Like, we're pretty far off of ideal policy.
So in one version of long-termism is let's just think about our children's world. And another is let's think about 150 years from now, which is pretty far in the future, but I feel more confident predicting trends out 150 years knowing I'll get some of them wrong. You then begin talking about a million years, a million years. What is long-termism to you?
Because the first thing you go out, obviously, the more uncertainty begins to bite. So how do you define it and how do you work with that uncertainty? From a value's perspective, I think that we should be caring about the whole future, but you raise an important point, which is the big obstacle to doing long-termist work is knowledge and it's very hard to predict the future. It's very hard to identify actions that will matter that long from now.
And so the vast majority of ideas we have would probably be wrong and will probably be overconfident about what will actually matter a million years, a billion years, whatever. So this is a huge challenge. And I think it's one of the downsides of being long-termist and one of the reasons that I haven't put my whole life into it and never will. But that doesn't mean that we're totally helpless.
It doesn't mean that we should just throw up our hands and say, let's just optimize for the next one year, because that's the same as optimizing for the next billion years. And climate change is an example of that, where if you're always focused on the next year, you might say, let's burn more fossil fuels because that makes us all better off today. But it's not some radical state of ignorance. It's not some giant, unknown question whether climate change is going to make the longer and future better or worse.
And whether it has the potential to make it a lot worse for a very long time is actually real possibility. I think as a long term is to do it well, you have to have taste and judgment and you have to know when you don't know something, which is almost always. And when you might be on something, that actually could matter for that period of time. That's not an easy thing to do.
And it's a pretty young idea. So I don't think we're nailing it, but I think someone should be trying it. So I know that you started out in a place that's frankly more like where I started out or maybe even where I am on these questions, which is a bit more critical of the idea of long-termism, more critical of the appeal of long-termism within the philanthropic circles you run in. But recently, you've put a lot of emphasis in it.
In OpenPills 2018, updated on cause prioritization, you wrote, quote, we'll probably recommend it, that a cluster of long-termist buckets collectively receive the largest allocation, at least 50% of all available capital, end quote. Now, I know those numbers are subject to change, but it speaks to the fact that long-termism is an area you decide to place a lot more emphasis on in recent years. So tell me a bit about your trajectory on that. How did you become more persuaded there and what did you become persuaded of?
Sure. So as CoCEO of Open Philanthropy, my job is to try to get ahead of the curve. My job is to look for ideas that are not only important, but also neglected. And so I'm always looking for what could be the next big thing that could matter for a ton of people that's not getting enough attention.
And I deliberately seek out people and ideas that can introduce me to things that might be that. And so through this, I've encountered the idea that this century, the 21st century, could be the most important century of all time for humanity. And a primary way that might come about is the development of AI systems that could cause a dramatic acceleration in science and technology, such that if you were to imagine a radical sci-fi future, a technologically advanced utopia, dystopia, or anything between or even maybe a world that's not run by humans at all, that's run by AI's with their own non-human compatible objectives, which we can get to. A lot of people think that kind of long-run future is possible, but the right kind of AI could bring it very soon, could bring it this century.
And once you think that you could be in that kind of century, now the timescales have collapsed. Instead of trying to make predictions about a billion years from now, we're trying to make predictions about the specific things that could happen in the next few decades that could matter for hundreds of years, thousands of years, billions of years. And so when I first encountered this idea, I think it all just sounded too wild and too out there. And I really kind of mostly stuck to the work I was doing.
But again, it's my job not to be too dismissive of ideas that could be extremely important and extremely neglected. So over the years, I've come to take it more seriously. And in particular, Open Philanthropies had a team really focused for the last several years on taking this thesis about AI in the most important century and poking every angle at it, looking for the weak points, trying to figure out whether this is really plausible. And we've gotten to the point where I'm not going to say that this is something I know.
I'm not going to say this is something that's going to happen. But I'm going to say it's a serious possibility that I think deserves a lot more attention than it's getting. And the most recent kind of change for me is I've been trying to get my thoughts straight. And so I've started my own blog called Cold Takes where I'm trying to lay out the case that we're in the most important century as simply as I can.
And more generally, I've been noticing that I've been taking on more unconventional views, more views that are important to what we're doing, but that are not widely held. And I think it's important for me to be writing up those views in a clear and simple way in public not only to help get clear in my own head about what I believe, but also because if I'm wrong, I want it to be easier for other people to encounter what I'm saying and to show me how I'm wrong. So this is a project I'm on now is taking these ideas that could be astronomically important that I've started to take quite seriously and continue poking at them by kind of putting them out there. Well, let's talk about that project.
We'll go deeper into these questions around AI in a minute. But first I want to step back and talk about the big picture of you of your eyes, like you need drums and horns when you say this, most important century series. One of the things you're really arguing there is that the future could be profoundly unlike the past in a way that it's true that 2021 is unlike 1900, but it's a lot like 1900. It's still human beings running around.
A lot of things are recognizable. A lot of the same religions you had Democrats and Republicans in 1900. There's a lot happening then that was quite similar, but you're arguing here that 2300 could be basically unrecognizable as a world. So it could be the basic case for that.
The case for why the future might be wildly different than the past. This is a big thing that has held me back from taking the most important century idea seriously for many years is that it just if it's true, it implies we live in this wild unusual time. And something I've learned is that if you just look at our time in full historical context, there's so many reasons other than AI to think that we do live in a wild time. So I think it's helpful to just kind of situate it in context.
First, there's the past. So the universe is more than 10 billion years old. Life on Earth is more than 3 billion years old. The whole thing of a species that's creating its own technology at all, like even stone tools, that's like millions of years old.
So that's millions versus billions. That's the blink of an eye on galactic timescales. And then human history is also just very packed into the recent past. So I think almost any metric you look at economic growth population, major technological milestones more has happened in the past few hundred years than in the previous several hundred thousand or several million years.
And I kind of point to this idea that there is a sort of acceleration that has occurred. Things have moved faster and faster. And if you simply project that acceleration forward and you say the acceleration continues or it's paused right now, but it comes back, in some ways things going so crazy and the next few decades being more eventful than everything that came before is a continuation of a trend, not a breaking of it. Then if you look at the future, I think there's other interesting observations about what a weird time we're in.
Today's level of economic growth is a few percent a year that doesn't feel that crazy to people, but it is not only an incredibly high level by historical standards. It's a level that doesn't look like you can go on forever. So if you try to project out like thousands of years of growth of even 2% a year, it kind of looks like you run out of Adams and the Galaxy and just can't do it. And so something has changed.
Hold it for a minute because I know the math on this, but it's very unintuitive. So as I remember the calculation run, 2% growth a year for 10,000 years will get you to a quantity that you basically can't run with materials of the Galaxy. But 2% doesn't seem that big. Yeah, I mean, the specific quantity is it's some multiple of the number of Adams and the Galaxy and there's very good reasons to think we would not be able to get even close to the edges of the Galaxy in that time because you're just constrained by the speed of light if nothing else.
So 2%, I mean, it's exponential growth. And if you just plot it out 10,000 years, exponential growth tends to be very unintuitive. Accelerating growth is even more unintuitive and even more explosive. And that is something we've seen accelerating growth or what's called super exponential growth.
We've seen it in the past. And if we see it in the future, yeah, I mean, things go to the moon very quickly and it's very unintuitive, but I do believe it's something that can happen. But why this century? What makes this century so important?
So three basic points. Point one is that the long run future could be just radically unfamiliar. It could be a radical utopia, a dystopia, anything between. Point two is that the long run future could become the near term future.
If the right kind of AI is developed to accelerate science and technology dramatically, and point three is that that kind of AI looks more likely than not this century. And then the bonus point four is that when you put the three together, a natural reaction is that this implies we're in a very special time and it sounds too wild to be true. But point four is that if you step out and look at our place in history, it looks like we're in a very weird and wild time for many reasons that have nothing to do with AI and so we should be ready for anything. I think actually just the idea of acceleration here is unintuitive.
I think if you know the basic growth story, it's been fairly steady 2% growth globally for some time. People hear a lot about great stagnation, they hear about wage stagnation. It's been very hard for a lot of countries to break out of middle income traps. It doesn't intuitively feel like we are in an era of globally accelerating growth.
So walk me through the accelerationist argument. Yeah, I think we're not in a period of globally accelerating growth and I don't think we necessarily will be again. We could just end up with a world that stagnates where growth slows like you're saying. The case for accelerationism would be, so the basic idea is if you look at most standard economic growth models, there's this potential for a feedback loop.
This is something that can happen. It's not something that's happening today. But you could have a feedback loop where every time you get more resources, more food, whatever, it leads to more people, more people have more ideas, more ideas leads to innovation and therefore more resources. So you get resources, people ideas, resources, people ideas and you get a feedback loop and that causes accelerating growth that can be this very explosive dynamic.
And it looks reasonably likely that this has described periods of economic history. The people refer to the Malthusian dynamic where people didn't get much richer, even though they were improving productivity because the additional resources would just go into more people. And so you see that pattern and what happened, what changed a couple hundred years ago or so is called the demographic transition where it stopped being the case that more resources meant more people. And of course, when people have a lot of wealth or have a lot of resources, they tend to just be richer.
It doesn't cause them to have more children. It doesn't exist. It's the opposite effect. They're fewer children.
Exactly. And so if that's the dynamic we're in, then yeah, you would by default expect that we will get stagnation. And I think that's a serious possibility that our long run future looks like economic growth has to slow dramatically. I think there's a lot of reasons to think that is our default.
The question is, is there a way to get that same function provided by people, provided by something else such as AI that can be straightforwardly produced by more resources? So today we have AI systems that are cool. They can be people at chess, they can transcribe audio. What they're not doing is they're not doing that innovation part of the feedback loop.
They're not creating new technologies autonomously. They're not advancing science and technology on their own because there's such limits to what they could do. And the question is, if that changes, if AI is developed, that could be as good as humans, doesn't have to be better at sort of pushing science and technology forward, then you get the feedback loop back and then you can get an explosion. So there are a number of things I want to question or describe more, but I want to first start at what I think will be the natural objection, which is something you didn't spend time on there.
In a population resources ideas feedback loop. Resources is a part of it. So I think it's got in the worst name recently. There is a broadly held view.
I hope that we talked about climate change already in the show that we've had a lot of growth that is used up a lot of resources in a way that for all the good it's done is also done a lot of harm. It's put us in a very dangerous climate situation. It has led to a tremendous amount of extinction of other species. The idea that you just accelerate growth from there, well, with what resources oftentimes growth does not create resources.
It consumes them, right? We're consuming fossil fuels that are finite on this planet, et cetera. How do you think about the resource constraint or do you not see it as a constraint? Like talk about that piece before we go into what would happen if you blew up the rest of the loop?
First, I just want to be clear that I'm not talking about this possibility as this is exciting and we got to do it. I'm talking about this is something that might happen for better or worse. And I think it could be very good or very bad. But in terms of resources, so humans right now are the only thing that sort of have ideas that sort of create new technologies that advance science and humans have a lot of needs.
And there's a lot of resources for humans that are really hard to replace. There's like only one planet that we really know of where it seems pretty doable for humans to exist for a long time. And once you have a different kind of technology fulfilling that part of the feedback loop, that kind of technology does not need all the same resources that humans have. So you need in order to run more ais.
You need more computers. Well, you need some things for that. You need metal. You need electricity.
You need cooling. But there's not that much that you need to run more computers. And actually, all the things you need are very abundant in space and all the things you need to get to space I think can be built with those sort of limited set of resources. So I think that's kind of the whole idea is that it's a lot easier to build more ais than it is to build more humans.
And so this loop could quickly get out of control if that dynamic changes. I want to put a pin in there's a lot of metal in space because I think it's actually an important part of your vision, a lot of people's visions that we should come back to in a second. But in terms of here, this is always a question I have about the AI unlocking tremendous economic growth question, which is oftentimes a constraint on an idea improving people's lives is material in some way or another. So for instance, there's a lot we could do with better analysis leading to better drug innovation, but actually a huge constraint in drug innovation is you need to run a lot of trials on human beings.
You need to actually test things in the real world. It slows everything down. It's a big deal. It's not clear to me in the first approximation, you know, where AI helps on that or how much it actually unlocks or in terms of things people might want.
Houses take would, you know, cars need to be built. It is true that the ais themselves would not need a lot of resources. But in order to get a huge amount of pressure on the growth number, which is measuring things economy actually produces and people consume, like a lot of this would have to be built. It's not going to all be like digital t-shirts in the metaverse.
And so how does AI create more resources as opposed to even potentially disaster using them up? I mean, should I look at resources as fixed and more growth will just consume faster? Should I look at them as like a pie you can expand? Like how do you how do you think about that part of the equation?
I think my first answer to the question is that if you assume that you can't translate these AI resources into human resources, it doesn't change the fact that we're looking at an enormously consequential development. So if you have, you know, this kind of dramatic feedback loop and a dramatic increase in sort of the reach of our world of our planet's artifacts, if there could be sort of this growing population of the eyes that is expanding throughout space, that's a very high-stakes situation for humans. If it turns out there's no way to convert all of that wealth into wealth that makes humans lives better, well, that's really bad news. It doesn't mean that this is a nothing burger though.
It kind of might mean this is really bad news. So when I talk about a giant acceleration in science and technological advancement, that doesn't have to be an acceleration in medicine. But it's an acceleration in some sort of general potency, some sort of resources, some sort of ability to make something happen and the less compatible that is with humans the worse, then it sort of becomes up to us. I mean, if this happened, could we harness it in a way that it was to humans benefit?
Could we take a very large supply of sort of digital minds or AIs having ideas and use that to make the world better? Or is it just going to turn into this sort of runaway train? And I think that's a question for us. Well, give me a concrete example.
Like, what's an example of a way if we unlocked this boundary we have on creativity and innovation, right? You have five billion AI minds coming up with cool ideas all the time. When you think about how that could lead growth to go vertical, what are the kinds of things you're thinking might emerge? What are the problems that we have might they solve?
I mean, it's all fine to talk about growth in GDP and AIs. But paint a sci-fi utopia or dystopia for me. Like, let's get tangible. Sure.
I'll start a little bit more tangible and then I'll get a little bit more out there. So for the little more tangible, we can talk about energy. Energy is definitely something that if you had a lot of minds and you had a lot of resources, you should be able to get a lot of energy. You should be able to get it very cheap and you should be able to get it in a way that isn't necessarily involving any greenhouse gas emissions or anything like that because there's just plenty of energy to be had if you had big enough and good enough solar panels.
That's something that I think could absolutely come out of this loop. And if we talk about health, I mean, you might be right that health is always going to be bottlenecked by human trials. But if you're able to do enough simulations, if you're able to have enough minds on the problem, you really could come up with a very large number of candidate drugs at once. You might have to wait a few years for the clinical trials, but it's sort of the sky's the limit in terms of how much you can improve health.
And so that's answer number one is health and energy. Well, that's an awful lot. We could go through other parts of the economy, but dramatic changes in health, lifespan, and energy would certainly be a big deal. Now, I'm going to go more to the wilder end of the spectrum and the more speculative end because I think it's important for people to not have their aperture too narrow, not be stuck on things that feel like today when we could be looking at dramatic changes.
So I do want to talk about the more radical end of things. It's very important not to have to narrow an aperture. It's very important not to insist on thinking about things that seem real today when we could be confronting a dramatic acceleration. So in the most important century series, I try to just give one concrete imaginable example that won't necessarily be the right one of how technology could get super crazy advanced and give us a huge amount of control over our environment.
I call the example digital people. This is very related to the idea of mind uploads. And it's the idea that you could have sort of people made of software, fully conscious beings. They don't have to be just like us, but they would be something that we decided were properly referred to as our descendants, some sort of digital mind that we would look at and say, I care about that digital mind, that digital mind matters.
And I identify with them, maybe I think of them as part of our civilization. And just from that, just from having either a digital running of a conscious human mind, and we could talk about whether such a thing would be conscious or without even being happy to get to that if you want, or just something else that we identify with for any reason, that could get extremely radical because any kind of digital being, it could be in a virtual environment, which means that it experiences anything that the runners of the environment wanted to experience. That could be unlimited good, that could be unlimited bad. There's no reason that digital people need to have any kind of health problems or any kind of life span limits.
And I'm not saying this is necessarily a good thing. There are very scary things about the idea of digital people. You can imagine a world where people are able to lock in certain values to their society forever because of the enormous amount of control you would have over those virtual environments. So I think digital people are something where if you just imagine it's possible, you visualize it to be concrete, you draw out the implications, it's very easy to get very quickly to a utopia or dystopia that just goes all across the galaxy and lasts forever.
Now, is that literally what's going to happen? No, it's an example of what could happen if technology advances enough and there's probably many more I haven't thought of. I understand the idea of AI. Sure.
Digital people is a slightly different concept. Yeah, it is. But that is something that we could do at a level of fidelity, that we should be thinking of them as conscious, having rights. People could simulate on, you have an interesting idea that one way digital people could really help is by accelerating social science.
Like you could see what happens. You make bunches that meditate through ours every day and others don't, but that you could have the fidelity in them that a simulation of digital people would be relevant to, I guess we call them real people, physical people. Give me the plausibility argument. I mean, there's many ways you might imagine having digital beings that we cared about for one reason or another.
But the plausibility argument I talk about just to be most concrete is just what if we could simulate everything that's going on in a brain. And brains are not these enormous rare artifacts. There's a lot of them. And there just doesn't seem to be a particularly good reason to imagine that we wouldn't be able to find some way of simulating a brain so that it is performing all the same important operations, all the same information processing, turning the same inputs into the same outputs, especially if you're imagining a dramatic acceleration in the population of AIs, which could be doing the science, which could be understanding how to run these simulations, which could be providing the computers we talked about, you could have as many as you want.
This is not something where I've like decided, oh, yes, this is definitely doable. This is just something that's more like, I don't see why not. I think if we were in this limit of having sort of infinite computing resources, I would expect this to happen by default, but maybe it won't. It's definitely an example of a way the world could go very wacky very quickly.
This loops us back to that economic feedback idea because this is something I think is really important in your thought. And it is worth a sitting in a minute, which is that you really identify the core constraint on growth as population. I don't think that's where people tend to think about it. They think of resources, they think about ideas, but in a lot of these different ideas, whether it's AI and does not have to be sentient, it can just be idea generating AI that never has a feeling of its own, whether it's digital people, simulations of us, whatever it is, you're imagining the world where you just completely blow the top off of the population constraint.
And tell me a bit about the population constraint as it operates in your thinking. Why do you see that as what would unlock so much potential for humanity? This is largely coming from the different models that economists are using to theoretically model growth, and they all have this kind of point of commonality, not all of them, but it's just very common. And OpenFlan3D did a report called report on explosive growth, where we're kind of very broadly surveyed the economic literature on how people model growth.
There's a lot of ways of doing it, but this feedback loop is just a very common property. It's a very common theme. And it's true that I think most people tend to think of population as essential input into growth, but I think that comes back to long term short term again, a cool paper that I recently wrote about. There's a paper by Chad Jones that just talks about what you think about long run growth if you're using what he calls the semi endogenous growth models.
And it does say that over the last hundred years, a lot of our growth has not been about population as not been about population growth. It's been about increasing education. It's been about reallocation of resources. But these things, they have limits to how much more than we can do in the long run.
It's kind of hard to see how you get continuing high growth or continuing improvement in living standards per capita if you're not having a rising population. So I think I think the idea of population is that humans are the only things that produce ideas, innovations, new technologies. You need them to do that. You need more of them.
If you want more of that, if ideas are getting harder to find over time, then you need even more of them over time. And so this becomes a very important factor. AI is moving fast across the enterprise, but without visibility, it's just chaos. ServiceNow turns that chaos into control.
With the AI control tower, you see all your AI in one place. To put AI to work for people, visit servicenow.com. Hi, this is Ashley. I live in San Francisco with my boyfriend.
We would love to officially share my near time subscription with separate logins. We both love cooking, loving in the kitchen, but I'm a 30 minute and under efficient dinner at Gurley, I want a sheet of meal. He is a elaborate. He wants to get into the storytelling.
I want to be able to save my easy meals and check off the ones that I've completed. And I think him having his own profile would be great. Ashley, we heard you. It's why we created the New York Times family subscription.
You get your own login and Mr. elaborate gets his. Plus room for two others. Find out more at nytimes.com.com.
So I do think taking the dystopic possibilities of all this seriously is important. Something that the sci-fi writer Ted Sheng once said to me stuck in my mind when I asked him about if he thought we'd ever create Ascension AI and he said, no, he didn't really think so, but maybe we could, but we shouldn't because far before we ever created Ascension AI that could kill us all, we would create AI that we could make suffer dramatically. When I hear about this possibility of creating these virtual environments, whether or not it is digital people or just us people, our people, I've always said the most likely dystopia in the next couple hundred years always feels to me like the ready player one dystopia where for vast amounts of the population, the actual opportunities in the world have just become so diminished that they are in a brave new world like VR rig and things have just become artificial in a thin way as a way of escaping from a grim reality. When you say you're not sure if this is good or bad, I think that the implication of what we talk about growth is it's all going to be great.
What are the things that worry you here? Things that worry me number one would be misaligned AI. So I take the possibility seriously and I know it sounds wacky in like a sci-fi movie, but I take the possibility seriously that things could just spin out of control. You've got AI's that can grow the population in a much faster way than humans and you've got AI's just running the world, setting up whatever they're setting up over the galaxy based on their own sort of random objectives that came in because they weren't carefully designed and are not compatible with humans.
So that seems like probably the thing that I feel best about saying let's really make sure we don't get that. That just doesn't seem like a human-friendly world in any way, shape, or form. Then what are other things I worry about? In general, I don't like the idea of rushing headlong into this world.
Of having a world where all the people who have inclusive and humanitarian values are focused on other issues, everyone working on AI is just someone who loves doing fun, sexy stuff and pushing tech forward, then we kind of rush headlong into this world where it's just it just becomes a race and it just becomes you have this dramatic acceleration and the people who are the least careful and the most eager to win the race end up running the world. Those are not the kind of people that I hope will end up running the world. So those are two things I worry about a lot and a lot of what feels central to me when I think about what do I hope for if this is the most important century is I hope we find a way to have more reflection than either of those implies. There's this term Toby Orde uses in the book The Press of his called the Long Reflection and it's the idea of having civilization take its time in a sense, use technology to give itself time to improve and to have moral progress and to not build some incredibly large powerful galaxy-spanning civilization before it's something that could be done in a fair inclusive thoughtful way.
So one of the ideas here that you talk about is also that we're in what I could think of as a quarter of existential risk. That until the 20th century, humanity hadn't really created technologies that could possibly wipe out all of humanity without a tremendous amount of effort. And it really would mean a lot of people going around individually killing everybody else. But then we created nuclear weapons and all of a sudden we could.
We've talked a little bit about the possibility of synthetic bio-weapons, misaligned AI, something people worry about on the score. This is a big point of Toby Orde's book. We are one of the first generations to really have the power to end human life. And we're not that wise.
Our technology has advanced far quicker than our governing institutions, far quicker than our moral intuitions, I would say. And so a point you make is that if you get to the other side of this technological era, if you can diversify humanity into other worlds, into other formats of being alive, like computers, maybe there isn't quite as much existential risk anymore. But between here and there, there's a lot. That also makes a reason this is an important period because we may be more protected from wiping out a civilization in a couple hundred years, but we're not now.
Yeah, that all sounds right to me. There is some future state where we've likely become very technologically advanced, where there's not a lot more advances around to kind of shake things up, where there is a great amount of control over the environment where we're able to be spread throughout the galaxy, have the stability that comes with that, where we might have the stability that comes with not having aging or dying be necessary or just having those take a very long time. So there's a lot of reasons I think that you might expect just a, the world to reach a state at some later date, where however things are, that's how they're going to stay, at least for very long periods of time. And I think that could be a very good thing or a very bad thing.
It just depends on what that state is. And that's what gives me so much vertigo about the idea of the most important centuries. It's just things speed up a lot and you get to some sort of radical future and the seat of it, the starting point, seems like it matters an awful lot. And it's this huge opportunity for things to go very well or very poorly.
And I don't feel like there's a lot of seriousness about what to do with that situation. So should we maybe just not? Should we maybe just not be trying to build AI? Maybe just not be trying to push some of these technologies forward?
Maybe just not be trying to grow as much as we try to grow? I think if it were up to me, we would be really slow and deliberate and reflective about it and sort of have some process for making sure that a lot of people were included in a serious reflection on whether this is a good idea before anything really dramatic and irreversible happens. It's not to me. I think this is coming.
I think the economic incentives to create this kind of AI are very strong. The idea of having the whole world just not work on it doesn't seem viable. I also think there are tremendous potential benefits from AI and from all technology, like I said, I think that the pattern of economic growth seeming like a good thing does look true over the last couple hundred years and probably is true over the coming decades at least. So I just don't know that it's realistic to go the way you would kind of go in a happy dream world.
And I think instead we have to take this as something that's coming our way and think about what to do about it. This entire vision for you actually rely here on this kind of AI, which is to say that if for some reason it just turned out that you couldn't create this kind of transformational AI, that it was never able to learn on enough real world data to create real world innovations that the computing was too difficult. I don't know. Something was found and it just doesn't.
There's a limit we hit and we can't go beyond that on creating computer intelligence. Does that feature just look pretty normal to you? It looks a lot more normal. It would definitely be a big change in my views.