Have SiriusXM in your car? Listen free from now until June 1st. Enjoy 100 plus channels of ad free music from every genre. Live sports, news, and must hear talk.
And the best part, it's completely free. Simply press the SiriusXM button to unlock your car's potential and take the best and audio entertainment for a free ride today. Offer ends June 1st, 26, exclusions apply. Visit SiriusXM.com.com.
free, listen for details. SiriusXM, your drive there could be way better. I'm Mr. Glenn, and this is the Ezra Glenn Show.
So before we get started, a bit of housekeeping, promotional kind of thing. There is a subreddit, a page on Reddit for this show. I'm not involved in it. We don't control it or moderate it or anything, but there are a couple thousand people who like talking about the show over on Reddit.
It's at reddit.com slash r slash Ezra Klein. And they've asked me to do an ask me anything next Friday the 11th. And I thought I would tell people this is happening in case you want to go participate in it. It'll be sometime in the morning Pacific time for me, but if you want to head over there, there's going to be an opportunity to ask questions and I'll come for a couple of hours and answer them.
So that is at reddit.com slash r slash Ezra Klein. So one of the strange things about living in the Bay Area is you'll be going about your business, you know, talking to people about politics and parenthood and how nobody can afford a house. And all of a sudden you'll run into someone who works in artificial intelligence. And you'll realize the world and what is important in it and what is coming in it looks entirely different to them.
Like they are living across a chasm in expectations from you. But to them, we are on the cusp of a technology that will be more transformational than simply computers in the internet. They'll be on the level of an industrial revolution. It maybe could create a utopia, could create a dystopia, could end humanity entirely.
And you can dismiss it. Sometimes I do, I don't have the impulse to. But these are smart people, I know them. And one thing that gives me pause is they are inside something I am outside of.
They are seeing inside a technological revolution that's closed to most of us. They're seeing how fast a technology is moving when, you know, I mean, I try to follow it and I have no idea. And so one of the things I've resolved to do this year is get a better handle on what I think of artificial intelligence, what I think it's going to do to the economy, to society, to our politics, what I think the political approach to it should be. And that means first in form is just understanding what it is, what it's doing right now.
There's an important distinction to make here. AI is not just Skynet. It doesn't mean sentience. It doesn't mean super intelligence.
To be honest, it's not clear what it means. There's an old joke in computer science circles that artificial intelligence is anything a computer can't do yet. So like before computers can be humans in chess, well, that would be artificial intelligence afterwards. Well, that's just machine learning, chess is a game, it's rules, you can just tell the computer the rules.
So there's always an expanding frontier over which, well, that is true intelligence and this is just machine learning. But broadly, we're talking about machines that can learn and act more autonomously. And this is not a far off thing. We're using them now for all kinds of things, from what ads Facebook serves you to where your bail is set after getting arrested for crime.
It is affecting your life, my life now. It is reshaping our economy and society now. And it's moving really, really fast. And even a lot of the people running these systems, they don't fully understand what they're doing.
And that's to say nothing of the politicians and regulators who are supposed to be governing them. So let's get started. There are going to be a few episodes around this theme in the coming months, but Brian Christian is where I wanted to begin. He's the author of the book, The Alignment Problem.
There are a lot of good books out there on AI right now, a couple of them from colleagues of mine at The New York Times. But Christians, in my view, is the best book on the technical questions of machine learning. We're in for a general audience. It's a very, very, very deep work on how machine learning works.
And it also ends up being a pretty deep look into how human learning works and the very fraught relationship between the two because that's the fear at the core of all this. The problems and the possibilities of AI are in a very deep way, the problems and possibilities of humanity. They are generated by us. The fear is that it will learn the worst of us, and it will take our mistakes and our dark impulses and reorder society around them.
And it will do so for the profits of a few. That's a really important part of this conversation that often gets missed. The business models of AI, the political economy of AI, it really, really matters. So I ask Christian to join me for the first of our AI episodes to talk about it.
As always, my email is asreconcho at endmytimes.com. Here's Brian Christian. I want to start with the concept that gives your book its name. There's just a very idea of an alignment problem.
And I'm interested to learn from you that it comes from economics, from commentary about capitalism. I actually think it's a useful place to start before we get to the whole artificial intelligence side of it. So tell me about the history of alignment problems. Yeah, so the term alignment gets borrowed in 2014 by the computer science community from the economics literature.
So going back to the 90s and 80s, economists were talking about, how do you make a value aligned organization where everyone's pulling in the same direction? Or how do you align someone's, this ordinance incentives with what the manager wants them to do? And obviously, there's a huge literature on how this can go horribly wrong. And it also connects to the parenting literature.
Every parent has had these experiences of, the example I love is that at University of Toronto economist Joshua Gans decided to get his older child, I think a dollar or a piece of candy I forget, every time they helped the younger child use the toilet. And he later discovered that the older child was force feeding water to the younger sibling so that they could use the toilet more times per day. And so I think this just points to how fundamentally human, this problem of incentives really is. And so this idea of how do you capture in a system of rewards or incentives or goals or KPIs or indices, what you really want in a way that's not going to lead itself to some loophole or some kind of horrible side effect that you didn't intend.
That is a much bigger human story than merely the history of AI. So there is a moth to the flame of super intelligent AI that will kill us all dynamic in this conversation. People like to sort of throw the ball away down the field and imagine this guy in that. But a lot of what is happening right now is we're building machine learning into things we currently do and predicting whether or not a violent offender will reoffend.
And so what kind of bail should they get or what kind of parole should they get? Or deciding whether or not somebody's going to be a good fit for a job. And so can you talk a bit about the ways that the problems you're looking at, the problems of aligning what we want machines to do and what they actually do are operating in the here and now, not just the far-flung future? Yeah, absolutely.
So this question of, are these systems actually doing what we want? Has this long history of course back to 1960, the MIT cybernetesist Norbert Wiener has this famous quote where he's talking about the Sorcerer's Apprentice. We all know it as the lovable Mickey Mouse cartoon where he tells this broom to fill up a cauldron with water and then ends up almost drowning. Wiener has this quote where he says, this is not the stuff of fairy tales.
This is coming for us. And the famous quote is, if we build a machine to achieve our purposes with which we cannot interfere once we've started it, then we had better be quite sure that the purpose we put into the machine is the thing we really desire. And this has continued through the early 20th century as the thought experiment of the paperclip maximizer that turns the universe into paper clips killing everyone in the process. But to your point, I don't think we need these thought experiments anymore.
We're now living with these alignment problems every day. So one example is there's a facial recognition data set called labeled faces in the wild. And it was collected by scraping newspaper articles off the web and using images that came with the articles. Later, this data set was analyzed and it was found that the most prevalent individuals in the data set were the people who appeared in newspaper articles in the late 2000s.
And so you get issues like there are twice as many pictures of George W. Bush as of all black women combined. And so if you train a model on that data set, you think you're building facial recognition but you're actually building George W. Bush recognition.
And so this is going to have totally unpredictable behavior. Similar thing, in criminal justice, you may think you're building a risk assessment system to tell you whether someone will re-offend or recidivate. But you can't actually measure crime. You can only measure whether people were arrested and convicted.
And so you haven't built a crime predictor. You've built an arrest predictor, which in 21st century America is a very different thing. And so there are many cases like this large and small. You see the same thing happening in the research community.
You think you're trying to build a program that can win a boat race. And you use this proxy incentive. You get as many points in this video game as you can. But maybe getting the most points involves just doing donuts in this little harbor and collecting these little power-ups forever.
And so each of these is in its own way, an example of this alignment problem. It turns out to be really hard to actually get the system to internalize the goals and the behavior that you actually have in mind. Can you talk through the story of Amazon trying to build this into their recruiting efforts? Yeah, so Amazon was building a machine learning tool to help them prioritize which resumes should get filtered through when they made a job opening.
And famously, they were rating their job candidates on a scale of one to five stars, just like Amazon users rate their products. And in order to do this, they were using what's called an unsupervised language model. But the basic idea is it looks at resumes for candidates that were hired in the past. And it says, what are the terms that tended to appear on the CVs and resumes of people who we'd hired before?
We'll just kind of upvote the resumes that look like that. The problem was that by default, this is going to perpetuate any kind of bias or prejudice that existed. So if your engineering department was mostly email, then you're going to discover, as the Amazon engineers did, that it is penalizing resumes that have the word women's in them. So if you played women's soccer or if you attended a women's college or whatever it might be, the system says, that doesn't look like the kind of language that has appeared on resumes of people we've hired in the past.
Therefore, we don't think you should hire this person in the future. And they ended up penalizing terms like field hockey or sewing or things like that that also kind of skewed towards female applicants. And even it went so far as identifying idioms that were more typical of male engineering candidates, like the use of the word executed. So it was giving you bonus points if you used this kind of martial speech in your resume.
Eventually, they essentially decided to scrap the project that it was kind of irredeemable. So one very tricky quasi philosophical issue here is whether or not this is in fact an alignment problem at all. So what you're basically saying is what happens is you turn these algorithms loose and they replicate the way our society actually looks. They look into Amazon hires and then they learn based on that hiring process.
And they say, here's you're probably going to hire an Amazon and says, oh, no, no, no, not us. That isn't how we wanted to hire at all. But on some level, it is how they were hiring. And maybe somebody who's less politically correct to use that language might say, no, the machine had it right.
And Amazon just doesn't want to admit what it actually does and what actually works for it. So these aren't alignment problems. These are, we're turning almost too powerful spotlight on where society really functions. But that's not a problem of the machines.
There's this adage that is famous in statistical circles. And it says, all models are wrong, but some are useful. And I think part of the danger that we have at the current moment is that our models are wrong in the way that all models are wrong, but we have given them the power to enforce the limits of their understanding on the world. So one example here being the self-driving Uber that killed a pedestrian in 2018.
If you read the National Transportation Safety Board report, you discover a number of interesting things. One was that apparently there was no training data of J-walkers. It was only ever expecting to encounter someone at an intersection or crosswalk. And so it just wasn't prepared for how to deal with someone crossing in the middle of the street.
The other thing you find is that it was using this kind of classic machine learning thing of every object belongs into exactly one out of n categories. And this particular woman was walking a bicycle across the street. And so the object recognition system couldn't make its mind up. It said, OK, well, I can see her walking.
No, there's definitely a bike frame. I'm seeing the tires. I'm seeing this little triangular support thing. She's definitely a cyclist.
No, we can see her walking on the ground. And each time it changed its mind, it had to kind of recompute from scratch whether it thought it was going to hit or not. And that's part of what led to the accident. So this comes back to this idea that all models are wrong, but we've now, in some cases, given them the ability to use effectively lethal force to ensure that the world conforms to their simplified preconception of what the world is.
If you live in a world where you don't think J-walkers exist and you can only be a cyclist or pedestrian, and you kill anyone who doesn't fit into that conceptual scheme, then the world does, in fact, come to resemble the model that you have in your head, not in a good way. So what are we trying to achieve here? Before we get to the question of how do you handle the problem of machine learning algorithms, creating disasters that we didn't intend? What is the promise of these algorithms?
They've been around for a little while now. We don't have supercharged economic growth. We've not cured cancer. Why is there so much promise and investment in this field?
I think there's broadly this idea that we can deploy human level expertise or intelligence at scale for kind of zero marginal cost. I think that's the broad idea that if we, not everyone can afford to go to a world class, germatopathologists to decide whether that discoloration on their shoulder needs to be looked at or not. But if you could train a model that's as good as the best germatopathologist in the world, then everyone with a smartphone could have access to that level of skill or that level of insight. I think that's the broad story in terms of why this technology is so attractive.
There's a well-intentioned societal motive behind some of this. A lot of the ways that hiring has been done in the past is just sort of through the social network of the people that already existed the company. But that privileges people with certain demographic attributes they live near the other people or they're in the same kind of economic class, et cetera, et cetera. And so there's this meritocratic idea of, no, let's create a job that's not posting and anyone in the world can apply.
But now you have to filter the candidates at a scale that you didn't have to deal with before. And so what do you do? Well, often they turn to machine learning. And as we've discussed, there's these kind of predictably terrible outcomes often.
But I think that's the idea in terms of your question. What are we really trying to do? Last summer, the coolest place in the house was in your freezer. This year, it's time to level up.
Reliance HomeComfort has over 160,000 five-star reviews for delivering the type of outstanding customer experience Canadians have counted on for over 60 years. Right now, don't pay for 12 months on a featured air conditioner or heat pump. Call on the experts that know how to beat the heat. Call on the products.
Conditions apply. See website for details. Hi, this is Ashley. I live in San Francisco with my boyfriend.
We would love to officially share my New York Times subscription with separate logins. We both love cooking, love being in the kitchen. But I'm a 30-minute and under efficient dinner at Gurley. I want a sheet pan meal.
He is very elaborate. He wants to get into the storytelling. I want to be able to save my easy meals and check off the ones that I've completed. And I think him having his own profile would be great.
Ashley, we heard you. It's why we created the New York Times family subscription to get your own login and Mr. elaborate gets his. Plus room for two others.
Find out more at nytimes.com slash family. So people are following the AI conversation at all. They're a couple big companies doing this. People hear about Google's DeepMind.
They hear about OpenAI. There's obviously a lot of work at Tesla. And then of course, also at Uber on how to do driverless cars. It's not going to be the case that every small hospital system develops its own remarkable AI player.
So it's an AI dermatology department. So is what's happening here that all a bunch of these different institutions are trying to create the AI interface that other people are going to rent out for their projects is what's happening here. There's just going to be a few people have it early on. And then other people develop it.
When we talk about this, who's going to control this resource? And how do others get access to it? I think this question really depends on the level of sophistication of the particular tool. So on the more spreadsheet end of the spectrum, I think, for example, if you look at criminal justice, pretrial risk assessment algorithms, many jurisdictions have rolled their own.
So Minneapolis rolled their own risk assessment in the 90s and made a note to themselves to audit it a year later, but then forgot. And then it was in operation for about 15 years before they even thought to check whether it was making accurate predictions or not. So those sorts of things happen. On the other hand, yeah, to your point, there is a certain kind of economy of scale.
At the moment, a lot of the most performant models are big. They take tens of billions of dollars to train. And so that's a certain barrier to entry. And it's a really interesting question.
Because if you look at the academic literature on AI safety, it's kind of premised on this almost obvious relationship of. There's a human called H who wants to do something. So he goes to his garage and builds a robot called R is R going to do what H wants. And maybe that's a useful way for framing some of the actual math.
But I don't think that's a useful way for thinking about the actual relationship we're going to have with something like Advanced AI. I think it's more likely that it's going to be like the Open AI API or like the user agreement that we have with iOS or Android, where we never really own anything. We're just kind of subject to the terms and conditions that can change at any moment. So I think that is much more likely from my perspective to be the kind of relationship that we have, especially early on.
But you spent a lot of time with these different companies. I mean, you read this book is a festival of like this researcher at DeepMind and this researcher at Open AI and this researcher at Microsoft and so on. What are they all competing to do? Like what are they what is their implied long term business model?
How do they spend billions of dollars winning this race to create some kind of general or at least very, very powerful AI? Well, that's a very interesting question. Because if you look at the kind of the narrative that's being told. And this is yeah, so just to use DeepMind and Open AI as an example, they both tell a story that's something like, we're going to solve intelligence and then everything will follow.
You know, once we sort that out, then we can cure cancer, we can solve world hunger, you name it. And it remains to be seen how the development of AI and in particular AI safety would handle something like a kind of late business cycle, you know, revenge of the bean counters where they say like, okay, and why are we spending, you know, 40 million dollars on this like generic thing that plays chess all day long? So, you know, it does explain why you get some of these contractions. And I am personally very, very interested to see whether safety research stays as robust in the next five years as it has been in the last five years.
But that's the idea. I mean, I think at some level, they're waiting to find the business model down the line. Well, that worries me to be honest. I want to hold here for a minute because it's something your book gestures at, but I always think it needs to be a little bit of a deeper conversation here because you've been talking about AI safety and for people who aren't fully into that term, that's around this alignment problem.
Sometimes it's like, you know, long term will the I kill us all, short term will do the things we wanted to do. But you have a good line at one point where you're imagining just AI that sort of follows us around and helps us make better decisions in the things we need to make decisions on. And you write these computational helpers of the near future, whether they appear in digital or robotic form, like we both, will almost without exception have conflicts of interest, the servants of two masters. They're a sensible owner and whatever organization created them.
In this sense, they'll be like butlers who are paid on commission. They will never help us without, at least implicitly wanting something in return. And so I imagine Google building, you know, through DeepMind, like the winner of the AI race. And I know how Google works.
And everywhere I go on the internet, Google is serving me ads that are built on my own personal data and that are trying to get me to either complete a purchase I've begun or seem interested in or trying to get me to make a purchase, I think, is adjacent to something that I would like. If you had much, much, much, much smarter AI that was much more integrated into my life that was built on an advertising model, you know, you're kind of entering a pretty nerve-wracking space of personal manipulation. And I don't hear this honestly talked about as much in the AI safety conversation. It's like much more like we're going to create this amazing thing, but what if it goes wrong, more than we're going to create this amazing thing?
And what if we do wrong? Yeah, I think this is extremely central. And it goes back to your highlighting of kind of the term alignment was originally an economic term that, you know, Google can build an AI that's aligned with the interest of Google Corporation, that may not be aligned with the interest of the end users or third parties that are affected without even using the software, et cetera, etetera. There's this question of what replaces like the right-hand margin?
Like for those of us that remember, you know, the internet of the early 2000s, right? There was this right-hand margin on every website that was full of ads. But you move from that to an oral interface where you're talking to Alexa or you're talking to your smart home speaker. What is the equivalent of this kind of right margin of space that we can fill?
Is it that you ask, you know, Alexa, what's the temperature? And it says, oh, it's 72. By the way, I thought you might be interested in tonight. There's a new show premiering.
Like, I don't know that people have the patience for that. And so there's this question of, is the advertising model going to survive the move towards these digital assistants? Is it going to have to be replaced with essentially like product placement slash commission driven model where you say to your smart robot of the future, like, just get me some toilet paper. And behind the scenes, there's been a huge bidding process for which toilet paper it's going to get you.
And at some level, maybe you care, maybe you don't. That's the kind of thing where I think a lot of the action is going to start to move to that. And so, yeah, you can really ask this question of, is this interface that I'm using actually working for me when I tell it what I want? Probably mostly not, right?
It's probably mostly worried about which kind of toilet paper is giving the best commission that week or whatever it might be. Or something deeper than that. I mean, I'm very interested in this question of the alignment problem between the end user and the owner, right, with the machine in the middle. Because another possibility here is geopolitical.
So we've been in a debate over the past year, too, in this country over TikTok, which is this remarkable social networking app that is owned by a Chinese company by dance. And there's been a lot of worries that maybe TikTok is spyware or what is it really doing. But something very central about TikTok is it it's underlying algorithm is amazing. If you look at analysis of why TikTok does so well, it's ability to intuit what you like through machine learning and feed it to you.
It's absolutely best of class. It needs a lot less information about you than a Facebook or somebody's other players do. Now you imagine I'm trying to make tremendous, tremendous investments in AI. Now you imagine that some of those actually pay off, you build some of that into TikTok.
And just on the margin, they're trying to make you like China more, which maybe is not even the worst thing. We've had propaganda efforts in this country forever trying to make people like America more. Why not use your algorithm on your free video app to serve up things that, you know, improve cross-cultural communication? But over time, this stuff becomes really, really out of alignment.
And it is even hard to know if it's going on. And I just don't really know, I guess what we do about it. Yeah, the question of what we do about it. I mean, there's two halves to your question, I guess.
One is kind of what's the end game of this? I am very curious to see whether, for example, Twitter and Reddit in particular, these kind of quasi anonymous text-based discussion forums, whether they can actually survive the ability to produce like site-specific propaganda at scale. You know, so we're now entering this era of large language models, things like Burt and GPT-3, et cetera, that can produce, you know, hand-tailored responses to a given comment thread that, you know, Whittily referenced the previous comments, but maybe have a slight 5% positive skew about, you know, pick your political party. I really don't know how the idea of like anonymous discourse survives.
And so that I think is a really open question. What do we do about it? I think partly there is a transparency issue and I don't know what the regulatory framework is going to be. But one of the things that I always want to know is what is the objective function of the company?
What is being optimized for? And ideally, you'd want to have some agency over that. You know, I mean, one of the things I like about Reddit is that there is this little dropdown where you can say, show me the most recent things, show me the most up-loaded things, show me the most controversial things. And you know, that's only a couple degrees of freedom, but it's something you feel like you have your hand on the wheel to some degree.
Whereas when I use other social networks, I'm very aware of the fact that my behavior is sending some kind of training signal back to the mothership, but it's really not clear what that relationship actually looks like. And so I don't know, for me, transparency is a starting point. Just what is it you were trying to do in the first place? Like what is driving the recommendations that are being sent?
Well, let's make transparency to levels because the transparency of these algorithms and transparency of what these machines are actually doing when they learn is a huge part of your book. But before we even get to that question of do we know what is happening in them, there is a question of do we have the right to know what is happening in them? I can't pop the hood on the Facebook newsfeed. They don't give me that option.
And they would say, and it's not a crazy thing to say, the Facebook newsfeed is our comparative advantage. The algorithm that feeds us is something we spent however much money on. You can't make us turn that over to the public. Do we need to think of algorithms in some kind of different class than we have thought of a lot of traditional forms of IP?
Yeah. I mean, in some ways it reminds me a little bit more of financial regulation where you have to deliberately make the regulation very vague because by the time the bill is actually passed, the technology has moved on. And so the only hope you have of any meaningful regulation is to kind of do it on the fly technically. I mean, that may be the case here.
You know, one of the other things Facebook may say is like, well, we can try to provide you transparency into our algorithm, but we changed the algorithm 20 times a day and we're constantly testing 1,000 different variations on any given user at any given moment. So what do you mean by the Facebook algorithm? You might have transparency one minute and then you refresh the page and now it's a completely different process affecting the results. So I think there are huge questions and I don't claim to have any idea what the regulatory foothold is here.
And I do think from the scientific side, there's been a lot of progress on the idea that you can actually constrain the model in a way that makes it intelligible to someone from the outside without sacrificing a lot of performance. So there's a really encouraging scientific story. How that actually rolls out into something that's user facing, I think is much less clear. Talk me through a bit of that scientific story because it is interesting.
I mean, five years ago, we were, I mean, all these things were more rudimentary, but we were much worse at figuring out what was happening inside of them. What got figured out and to what degree such that we can now know better, like what it is a machine is alert when it is producing an output. Sure. So I guess one place to start is if you think about the model, it has kind of three parts.
There's what are the inputs, what goes on in the middle and what are the outputs. And so one way to make a model simple is to have fewer inputs. One way to make it simple is to have less stuff going on in the middle. So one of the big advances that we've seen in machine learning since about 2010, 11, 12 has been the use of this technology called Keep-Grown Networks, which basically just is these simple rudimentary mathematical elements that kind of sort of resemble what goes on in a neuron.
It's like they have a bunch of inputs and inputs are just numbers. It's like one, zero point five, whatever. You add them up and then if they're greater than some threshold, you output some other number. Otherwise, you output the zero.
And it turns out that if you have tens of millions of these things stacked into layers, they can do essentially arbitrary tasks. They can tell cats and dogs apart. They can tell cancerous and non-cancerous lesions apart, et cetera. But there's this real inscrutability.
So even if you could pop the hood, you would see like 60 million of these rudimentary things that all have slightly different thresholds of when they output and when they don't. And so the question is like, is that level of transparency actually giving you anything? So that's been, there's been two pushes. One push is do we need to use this inscrutable technology or can we use simpler models, you know, your more classic 20th century statistics?
Then the other push has been, okay, for certain applications, let's just say you have to use a giant neural network, can we actually visualize what's happening with the interior of this network? Can we actually see, for example, that, okay, layer one has detected this edge or layer two is detecting this pattern in this part of the image? You're talking here for visual recognition. Yeah, as an example.
Yeah. So that's the kind of the two front attack that's being made is making simple models competitive with the complex models and making the complex models somehow more intelligible. I want to put the simple model question to the side because I take it as almost axiomatic that if these different players like DeepMind get where they want to go, we're going to be using some very, very, very complicated models and some very, very complicated systems. And if we get to the kind of general artificial intelligence, it's not going to be a simple model where we have it already.
So at some point we're going to need to know what these programs searching through all of the information humanity has ever been able to generate are finding. And we know that when they start searching, they find things we didn't, right? We know that say the model that is now the best AlphaGo player in the world, realize different things playing against itself in AlphaGo than human players had ever realized. And so how do we see what the model is seeing?
Yeah. One way that you can do it would be this idea is called perturbation, which is to say, you know, to use your example of Go. You start with the Go board and then you iteratively add and subtract stones to every location on the board and you see which of those perturbations have the biggest effect in what the model thinks is going on. And you can say, oh, well, when I remove this one stone here at the bridge between these two clumps, it totally changes its evaluation of who's winning.
Therefore, I can kind of infer that it's quote unquote focusing on this area or that this area is kind of salient. And that type of process can be really helpful. There are many, many horror stories of researchers finding that the model was essentially focused in completely the wrong area. So a group of dermatologists built a system to determine whether these, you know, marks on your arm are cancerous or not.
But when they use these kind of perturbation or saliency methods, they discovered that actually what the model is looking for is the presence of a ruler next to the lesion because the medical textbook images had a ruler next to them for scale. So I think that the ruler is the cancer, things like that. So that ends up being really helpful. The other thing that you can do with Go programs as being an example is to run the model forward and say, okay, what do you think is going to happen if you take the action that you're planning to take?
And that's another way to get a sanity check of just if we kind of look through the crystal of all, does the models sense of what's downstream actually track with what makes sense to us? Last summer, the coolest place in the house was in your freezer. This year, it's time to level up. Reliance HomeComfort has over 160,000 five star reviews for delivering the type of outstanding customer experience Canadians have counted on for over 60 years.
Right now, don't pay for 12 months on a featured air conditioner or heat pump. Count on the experts that know how to beat the heat. Conditions apply. See website for details.
Hi, I'm Solana Pine. I'm the director of video at the New York Times. For years, my team has made videos that bring you closer to big news moments, videos by times journalists that have the expertise to help you understand what's going on. Now we're bringing those videos to you in the watch tab in the New York Times app.
It's a dedicated video feed where you know you can trust what you're seeing. All the videos there are free for anyone to watch. You don't have to be a subscriber. Download the New York Times app to start watching.
So we've been talking here about what happens once a machine is learn something. But a lot of your book is about how we are learning to help machines learn and the places we're taking inspiration from on that. And a lot of where we're finding some inspiration is actually us. So can you tell the story of dopamine and what we have learned dopamine is for in the human mind?
Yeah, so in the 70s and 80s, we were learning a lot about the dopamine system and we were developing the actual technology to monitor dopamine, individual dopamine neurons in real time and watch them spike. And it was producing a pretty mysterious story. We could see a monkey reach into a little compartment and find a piece of fruit and boom, there'd be this dopamine spike. But the fifth or sixth time the spike would go away.
And so what was going on here and to make a long story short, you know, there's this question of is dopamine encoding our sense of reward? No, not exactly. Is it encoding our sense of surprise? No, not exactly.
So what's going on? Because we know it's related to those things, but we can't really pin down what this signal actually corresponds to. In parallel, there had been kind of this movement within the computer science AI community called reinforcement learning. And the basic idea was let's build systems that can learn to take actions within an environment to get as many points as possible, whether you define, you know, you define the points however you want, right?
So playing chess, you want to capture pieces or whatever it might be. And one of the methods that was successful in the 80s for solving this problem is what's called temporal difference reinforcement learning. And the basic idea was, you know, you take an action, you don't know if it's good or bad. You might not have to wait till the end of the entire game in order to know whether you made a mistake.
If you suddenly lose your queen, you know that something went wrong. You don't have to wait till you get checkmated, you know, 30 moves later. And so you can learn from these minute to minute differentials in how well you think things are going. It creates kind of an error signal.
And you can learn, you know, if you think about trying to predict the weather. On Monday, you predict the weekend weather. And by Tuesday, you have a slightly different prediction. You don't even have to wait and see if you're right.
You can just update and say, okay, my Tuesday prediction is going to be slightly more accurate than Monday. So my Monday prediction should have been a little bit more like that. And these are called temporal difference errors. It's kind of, you're learning what a guess should have been from a subsequent guess that you make.
Okay. So this was all happening, you know, in the computer science departments. So these kind of neurophysiology data land on the desk of some of the computer scientists, in particular Peter Deann, who had kind of crossed over into the Salk Institute and was doing some neuroscience work. And when the computer scientists looked at it, they said, oh, this is the brain's doing temporal difference learning.
It is learning a guess from a guess. Suddenly, when you find a piece of food that you weren't expecting, you know, life just got a lot better than you thought it was going to be a moment ago. But if you come to realize that fruit's always in the box, then life is always about as good as you currently expect it to be. So there's nothing more to learn.
I just think this is a remarkable story for a couple of reasons. Primarily, it is this idea that computer science and cognitive neuroscience are engaging in this dialogue, this kind of feedback loop where each is now informing the other. And it also, I think, tells us a story about AI that we are, in my view, this gives us some evidence that we're not just merely working out some engineering hacks that happen to solve video games or whatever, but that we're actually discovering some of the fundamental mechanisms of intelligence and learning. We've kind of independently found the same mechanism that evolution found.
And in fact, there are many different kind of evolutionarily independent temporal difference mechanisms. If you look at Bs, if you look at, you know, different species, we really are on to, in my view, the philosophical pater of artificial intelligence, which is to figure out like how our minds work to begin with. So I love it. So basically what is being said here is that dopamine is a way of updating our expectations about the world.
Yes. That we don't feel good because things got better or worse. We feel good because we are now projecting things to be better or worse. And you're saying the book something that has left me thinking about it for quite a while, which is that this helps explain the idea of the hedonic treadmill, the idea that people get used to winning the lottery, they get used to losing a limb.
Why does help explain that? There is this funny connection between the dopamine system and the subjective experience of pleasure and happiness. You know, so this is obviously a major front in philosophy of mind slash neuroscience is how do we link the physical mechanisms in the brain that we can observe to the feeling of being happy or of liking something. And to the degree that dopamine is part of that story, it tells us that it feels good to be pleasantly surprised about how promising your life is.
But because you are using that signal to learn, you know, it's like you can't be permanently pleasantly surprised. Like eventually you're going to learn how to actually make accurate predictions. And so some of that pleasure goes away. And so for me, it tells a story about not just the hedonic treadmill as adults, but the way that evolution has given us this really general purpose learning mechanism that when you're one year old, you know, or six months old or whatever, waving a hand in front of your face.
At first, it's really delightful because you don't expect what's going to happen or you push something off a table and it falls on the ground and you're delighted because you had no idea what was going to happen. And eventually, right, you need to get your kicks by playing sports or by writing academic papers or writing books or whatever it is. And it's I think pretty remarkable that there is this general purpose, take the light in the ways that your predictions are wrong, but also improve your predictions, right? And this kind of sets up this whole trajectory of our life in a way.
But one thing that made me reflect on is the way that we have alignment problems with ourselves. Oh, yeah. So take, you know, the dopamine function you're talking about there. One way to describe the hedonic treadmill is that the things we believe in advance will make us happy don't end up making us happy.
So here we are, like, wandering through our lives telling ourselves that if we just like work so hard and get to this point, we're going to be happy and then we're not. And now dopamine is not doing something wrong. Like it's, you know, optimized for fitness and all the other things evolution wanted for us, which is not always happiness. But there's a funny way in which we're sitting here talking about how it's hard to create the correct reward functions for machines.
But we also don't even really understand day to day how to create the correct reward functions for ourselves. And so we're constantly doing things like every time I pull out my phone and open up an app because I'm tired. They don't actually make us happier, but we somehow learned, you know, give us a little hit of dopamine because well, maybe something is going to happen now that is a little bit better than being, you know, bored and exhausted right here, you know, even though what usually happens, I get annoyed at everybody fighting on Twitter. And so there's a way in which there's a lot of pathos in us trying to teach, teach other, you know, being sentient or not organic or not, how to think and how to live when, you know, day to day, I'm not sure we're so good at ourselves.
Yes, I think there's something really deep here, which is that, you know, you can think of humans relationship to evolution as having this alignment problem where there are certain things that evolution quote unquote wants us to do that will make us, you know, fit to stick around in an environment. That process is really complicated and hard to directly encode into the motivations and desires of people. So instead we get this weird reward function or these weird sets of incentives where, you know, we want to eat chocolate and have sex and open our Twitter on our phone and check our notifications and all the things that we actually want or are motivated by. And this really is the problem of reward design, right?
Like evolution has designed these rewards and we've in some ways over optimized for them. And I guess the, you know, part of the human condition is realizing that you have some degree of agency, you know, just because evolution wants you to propagate your genes and do this and that, like you don't have to, you have some degree of agency over what your own goals are. And I think that's interesting from a parenting perspective, right? So one of the things Alison got into talks about as a parent is that, you know, being a parent is not like building an AI system because you don't necessarily have the, in mind at the beginning the notion of what you want your children to want.
I mean, maybe you do it within a certain parameters, but you also want to give them the leeway to become their own people. So yeah, this question of how to behave with respect to set of rewards that are to some degree, kind of hard coded, but you also have kind of control over your environment. You can sort of shape your own reward function to a degree. I think this is, yeah, in transict to the human condition, absolutely.
So one of the things that the research here seems to be doing is giving AI researchers more respect for parts of the human reward system that seemed softer, weirder, more idiosyncratic, more fuzzy. And one of them is very much curiosity. Yes. Talk about what we've learned from trying to get computers to play Montezuma's Revenge.
Yeah, this is one of my favorite stories in AI. So there's a team that came together around 2013 to 2015 to try to build an AI system that could be not just a single Atari game, but every Atari game using the same generic architecture. And they managed to achieve, I think, some pretty incredible results. It was like 25 times better than a human at, you know, video boxing.
It was like 13 times better at pinball, et cetera, et cetera. But there was one game at which this model scored a total of zero points. And that game is called Montezuma's Revenge. And so there is this question of why was this one Atari game so difficult for this AI system to beat?
And the basic answer is that it has what's known as sparse rewards. So most Atari games, you can essentially just mash buttons until you get some kind of points. And for an AI system, that's enough to bootstrap the learning process. And you can say, okay, how did I get those points?
Let me do a little bit more of that in the future and so on. But in Montezuma's Revenge, you have to execute this huge sequence of really precise movements. Any mistake basically kills you. And only if you do this huge long chain of things correctly, do you get even the first points of the game?
And so the whole premise of this learning algorithm was that you would mash buttons until you got points and then figure out how to get more. But how do you learn if you can never get the first points to begin with? And this is a riddle that is, I think, wonderfully solved by babies. So the computer science community starts looking over the fence at ideas from developmental psychology because, of course, human beings play these games with no problem.
So there's something going on that enables us to understand how to play these games. We've known since the 60s that infants have this really strong novelty drive. And psychology is sometimes known as preferential looking. So if you show an infant a toy and then an hour later you give it a choice between that toy and a new toy, they almost always pick the new toy.
And this is such a bedrock result that it's used as a way to study memory and perception and things like this in basically newborns. So there's a very fundamental reward, essentially, that people get from seeing new things. The idea was, what if we just plug this novelty reward into our video game system such that we treat the encountering of new images on the screen as literally tantamount to getting in-game points, just as good as getting in-game points. Suddenly when you do this, the program has this kind of human-like drive.
It wants to explore the limits of the game. It wants to go through the lock door just to see what's on the other side. It wants to climb the ladder and jump the fence. And that's what it takes to beat this particular game.
So that, to me, is just another one of these wonderful convergences where some of the insights that we're getting into these very fundamental drives in human intelligence end up being imported in a very literal and direct way into AI software and then suddenly it can do things that could never do before. Do you believe we're going to get super-intelligent general AI or do you believe we're going to get is sort of like kids with sub-aut-like capabilities and certain things that we need? I think at the limit, I see no fundamental principled reason why we're not going to get some kind of super-intelligent general AI. The question is, what does the road there look like?
And I think the road there does look like these weird sort of sub-aut-like grown-up kids. You can think about GPT-3. So GPT-3 for people to know that is OpenAI's predictive text artificial intelligence platform. It's got a lot of buzz.
It's one that a lot of people are able to use. So that's a big place where a lot of people have begun to see how AI can work. It's essentially like someone who has lived their entire life in a windowless room with an internet connection and has read everything that's ever been written on the internet but has no idea what anything actually is beyond how it's spoken about. And it turns out you can fake it pretty well.
So that eventually some of that ignorance will catch up to you. So for the foreseeable future, a lot of what I'm worried about with AI is how we're going to accommodate systems like that. Because to even, in some ways to even get to worry about the super-intelligent AI, we have to survive the kind of savant grown-up children AI, right? And that's the story of the source of apprentice.
It was just this animated broom that knew nothing except how to pour water and that was dangerous enough. So I gave a version of this question to Ted Chang, the science fiction author, which was, you know, does he think we're going to get super-intelligent AI? And he said, no, not really. We're not going to get sentient AI either.
Maybe we could. But should we? And then he said, absolutely not. And I thought his reasoning was interesting.
I was thinking about it while I was reading your book, which is he said long before we got sentient super-intelligent AI, we would have AI that could really suffer. And given how human beings have treated animals, given how they've treated each other, given how they treat machines, we would make this AI suffer on a tremendous level. And as I read your book, there's a pathos to basically every program you describe. We have every time embedded it with an unbelievable want for something, a want to see new screens and Montezuma's revenge, a want to get points in a video game, a want to be able to fulfill whatever question has been posed to it, a want a want, a want, or creating these little desire machines.
And these are often things that it can't do. We're often things that we're going to lose interest in it doing. And I don't know at what point, I know philosophers think about this. You have to think about the moral weight of this, at what point for a program to not be able to fulfill its wants, it is feeling pain.
But it does strike me as well before we're going to have things that are so intelligent. We have a lot of sympathy for them. And I'm curious how you think about this question. It's a great question.
There is a computer science research group that has, I think somewhat tongue-in-cheek title of people for the ethical treatment of reinforcement learning agents. But there are people who absolutely sincerely think that we should start now thinking about the ethical implications of making a program play, you know, Super Mario Brothers for four months straight 24 hours a day. You talked about one that did Super Mario Brothers and it's just like caught in this game that has no more novelty and it's a novelty-seeking robot. And I thought it was so sad.
Yeah, it just learns to like sit there because it's like, well, why would I jump across this little, you know, pipe because I'm just the same old shit on the other side. Like, well, I might as well just do nothing. I might as well just, you know, kill myself. And there have been reinforcement learning agents that because of the nature of the environment, essentially learn to commit suicide as quickly as possible because there's like a time penalty being assessed for every second that passes that you don't achieve some goal and they can't achieve it.
So they're like, well, the next best thing is to just like die right now. And again, it's like we're somewhere on this slippery slope. I mean, there is this funny thing for me where the more I study AI, the more concerned I become with animal rights. And I'm not saying that AlphaGo is equivalent to a factory farm chicken or something like that necessarily.
But going back to some of the things we've talked about, you know, the dopamine system, some of these drives that are, you know, the fact that we are building artificial neural networks that at least, you know, to some degree of approximation are modeled explicitly on the brain. We're using TD learning, which is modeled explicitly on the dopamine system. We are building these things in our own image. And so the odds of them having some kind of subjective experience, I think, are higher than if we were just writing, you know, generic software.
This is the huge question of philosophy of mind is like, are we going to know if we manage to create something with a subjectivity or not? I'm not sure. But these questions, I think, are going to go from seemingly crazy now to, you know, maybe on a par with something like animal welfare by the end of the century. I think that's not a crazy prediction to make.
Yeah. And then you add in the fact that you can create, it's not literally an unlimited number because there's computing power associated with this. But at some point, the idea is it will be simple and not computing power cheap enough that you can create marginal AI agents at very, very low cost, right? That is how you all of a sudden get all these, you know, super low cost human level labors.
And it does seem a little scary. I've given a lot of attention to this question of what would AI do to us, you know, including well before super intelligent, right? Just putting people out of work, that kind of thing. And very little to the question of what would we do to it.
But I don't know. I read your book. And as you say, somebody's story say already make you feel terrible to hear, like the ones of the, the eye just killing itself because there's a penalty to doing anything else at this point. And the idea that we wouldn't know when it was feeling pain, which seems very plausible to me, is pretty profound.
So I don't know. I don't know what point that should actually affect what kind of research we're doing. Yeah. I mean, we have, we have a lot more options than we do with human welfare and animal welfare.
In terms of, for example, if the agent is this novelty seeking agent that then gets really burned out and bored, could we just like wipe its memory so that, oh, wow, you know, everything is new again. And everything is delightful all over again. And it's sort of living in this weird kind of 51st States environment. Is that itself unethical or is that sort of the only ethical thing to do at that point?
I mean, it gets pretty head spinning. I think there's also this question of, you know, will there be an ethical imperative to make models simple enough that they don't have ethical standing? And I mean, there's a joke that computer scientists make about, you know, you have this household robot and if you pay extra, you get the version that doesn't have a subjectivity. So, you know, it's not suffering, but you know, by default, the cheap one, I couldn't afford to add that in.
So it'll do what you want, but it won't like it. This whole, I have to say this whole part of the conversation, at least we feel real chilled. Yeah. And like, oh, we're just going to start mind wiping our robot slave helpers because then, I mean, I'm not the most red up in science fiction of anybody you could possibly talk to in a day, but I've read up enough to read a few stories about ideas like that.
And they don't, I wouldn't trust us with that kind of power in general and having, you know, creating a new class of possibly suffering servant workers, you know, but who we've paid nothing for and who we don't really think. I mean, given because I am an animal rights person and like what we do to chickens is really appalling. I have no illusions that there is some limit of suffering that we would inflict on a species that we have enough justification to not care about, but whose labor we find useful to us. Let me ask you about the more near term cost of this for four human beings and people talk about more, you know, entry-ang, ran for president and some degrees running for, for mayor of New York on the idea that machine learning is going to, and automation will put people out of jobs.
It has a little bit of that quality that the internet had for a while where you can see it everywhere but the statistics. Machine learning algorithms computation automation has gotten way, way better in recent decades. It has obviously put some people out of work, but we do not have a significantly higher rate of unemployment than we did a few decades ago. There are some changes for labor force participation, but not really gigantic ones that I think you can trace to automation.
So why don't we see a hundred fifteen percent unemployment from what we can do now and will we? To some extent, you know, I'm partial to the idea that we sort of create the problems we then have to solve, like anything that makes email easier to send tells you that it's going to solve the problem of email because you're only thinking about it from your own perspective. It's going to make it easier for you to deal with your inbox. But if everyone has that program, then there's just more email and everyone's still spending the same amount of time.
So it's a treadmill. There are other treadmills like that. I do think that AI has been a big part of the story of inequality that, you know, if you broadly take the lens of society in the Marxian view, right, the struggle between labor and capital, AI is the labor that doesn't, it's not human, it doesn't need it wage, it doesn't advocate for itself. And so I'm somewhat sympathetic to the idea that we're permanently tilting this millennia long tension or at least centuries long tension in favor of capital.
And there's a lot of deep psychological questions to think about, you know, the political rhetoric around the economy is framed in terms of jobs and rather than these more primitive things, like good health, good education, whatever, are jobs going to be the salient rhetorical framing of those things going forward? I'm not convinced that that's the case. I will be very curious to think about what it means for someone to be in the economy when most of the things that they can do are able to be done by machines better, right? So you think about Amazon Mechanical Turk, employees a ton of people and the tagline of Amazon Mechanical Turk is artificial artificial intelligence, namely people.
So dystopic, right? I mean, it's this kind of so in the green, you know, it's people, but almost in its very premise, like those are the jobs that are ready to get automated away. So I think it's interesting to think about what it means to contribute economic value in that kind of society. Right now there are a lot of people whose economic value comes from like what they can see, like their ability to process information visually or their ability to sort of manipulate objects with their hands, right?
If you think about sewing, like we still don't really have sewing at scale, even though we have all this advanced manufacturing for steel, et cetera, et cetera, what would it mean to have robots that are dexterous enough to sew? I don't have an answer to that personally. I mean, I think these questions of does AI kind of lend itself towards this kind of, you know, rich, rich or rich type scenario? In my view, yes.
And so I think we have to mitigate that probably politically, but those are huge questions. I think they're open questions from my view. But so it sounds to me like what you're saying is that the question directly is not jobs as much as it is dignity and status that you can imagine a world where AI is doing quite a bit. People have jobs.
They're just a little bit ridiculous because you're operating on the margins of algorithms you don't understand or you're just sort of mopping up behind them. And then particularly if you add in that the people who only AI are getting all the money out of this, so the people with these now kind of crappy jobs also don't have this other generator of status and dignity, which is money in our society, then you have a real social standing problem. And so one of the questions is like, do we understand? We can give a lot of things dignity.
We're doing 500 years ago. What I do for a living, which is talk to people and write some stuff about politics on the internet. I am given a lot of status by that in our society, but compared to feeding people at times when people needed to be fed, it's not that useful of a role. And there are a lot of things like that.
I mean, we like choose to have some of these roles are in view to dignity, but they really do depend on how we see them socially, how much we like give cultural respect to people and then how much we actually pay for them. You know, teachers get a lot less dignity than I think they deserve compared to say investment bankers because they make so much less money. I think if teachers were paid $500,000 a year, the money combined with the obvious utility of the role would make them like society's most honored people. Yeah.
And I'm also very struck by a lot of the things we do for pleasure are the very things that we have sort of automated away. Like for example, the past times of the upper class in Detroit, England were, you know, fox hunting and gardening, essentially hunting and gathering, right? That's like we've built this entire civilization so that you don't have to hunt and gather, but then you know, the people with the most privilege and the most leisure want to hunt and gather for fun because that's at some deep level like what we are built to do. So yes, I think there's also questions of status from the perspective of we're now, is now comparing themselves to the entire global population.
I remember I went to a panel at UC Berkeley a few years ago where there was, it was Jared Lanier, Neil Stevenson and Peter Norvig and they were sort of talking about some of these like long term economics of AI things. And Peter Norvig said, you know, it used to be the case that if you made pizza and your pizza was 20% better, you might get some super linear amount, you know, you might make 30% more money. But nowadays, if you make an email client that's 20% better, you make all the money and everyone else goes bankrupt or you know, whatever it is. We just need to sort of fundamentally rethink.
Yeah, these questions of, I think, standing, right? Like if it used to be the case that you could be the best guitar player in a 10 mile radius, and that would make you really cool. And nowadays it's like, oh, well, you're only the third best guitar player. So why would I watch your YouTube videos when I could watch the best?
I wonder if we are going to see a kind of willful parochialism come back as people realize that there are advantages to being, you know, essentially the big fish in a small pond. I don't know exactly what that looks like. One thing you might say about society as it currently exists is we've given status to ridiculous things, maybe because we need to keep people busy, maybe because capitalism rewards somewhat absurd things sometimes. But however it is, like the way we actually attach dignity to roles and then train people from the time they're born to be like achievement monsters trying to get through those roles is also not a great way of doing society.
And so maybe it'll be the case that at some point in the kind of post-scarcity future, you have automated agents doing a lot of the fundamental work of the economy and people can concentrate on things that say the classical philosophers would have thought are closer to the good life, things that John Maynard Keynes thought we'd be doing when he imagined how rich we'd be, which is like, we'll be painting and thinking about philosophy or even just more, I think, pros-aically spending time with our families and going to the park and playing sports with our friends and having a drink with our buddies. And there'll just be more time to enjoy being human and that won't be looked down upon because the reason it is looked down upon is we have needed to make that a low status, low class activity in order to keep everybody very engaged in this huge economic machine we want to feed. Yeah, I mean, I'm very sympathetic to Keynes' vision of like, you know, in hindsight, you're like, well, what went wrong, right? Because we really were on track.
And I think about the promise that technology offers society in the broadest terms is to make people happier. But when I think about, am I happier as a function of having been born in the 1980s than if I had been born in the 1880s? I don't think so. I mean, I have better dental care.
I'm not worried about like an abscess tooth or something like that. But broadly speaking, like I care about my family, my marriage, my friendships. I want to do interesting work and write books, which I could have done a hundred years ago. Viewed from that perspective, technology has like surprisingly little to offer.
I don't think it's bringing a lot to the table in terms of addressing the fundamental things that make people happy. I mean, relieving, you know, the sort of creature comforts and the physical drudgery associated with them, I think is huge. But like we've been past that threshold for several generations at this point. And I think people are getting less happy rather than more.
Well, doesn't this speak to a possible alignment problem that we were getting out in human beings earlier? I think there's an endless view that we lift the condition of scarcity. Well, then finally, people will have what they need to be happy. But if you believe that we evolved with our reward function, right, our dopamine system and all the rest of it is optimized for getting us through scarcity, for surviving and reproducing conditions of scarcity.
Then its absence drives us a little bit crazy. We're like the machine trapped in a system with no novelty. Like we need to find something to keep ourselves busy because, you know, otherwise we just get listless and a bit lame. Now, I don't fully buy that.
I mean, there's a very interesting discontinuity in research where people are very unhappy when they're unemployed, but if they then just shift into retirement during that period, they get happier because like the status of being retired is a much better cultural status than being unemployed. And there have been many, many human societies that have not worked on, like neoliberal capitalistic or for that matter, like modern, communist or socialistic thought, right? Hunter Gather has had a lot more free time than we do. So there are different ways of constructing a society.
But I do think there's something possible to the idea that when you say technology is built to make us happy, fundamentally what you're saying is it's going to lift scarcity. And it may not be that humans beings are happier, at least beyond a certain point, in a condition of less scarcity, or at the very least, that that condition takes a lot of adapting to and culturally reworking around in order to get the most out of it. I want to offer a complimentary story to that, which is I think as some of this work on the dopamine system and, you know, intrinsic novelty seeking behavior and inference shows us not everything is about scarcity. There's something pleasurable about just visual unpredictability.
It's why we like looking at screen savers is why we like looking at campfires, where, you know, moving water. And I think there's something valuable there too. And for me, the natural world offers something that's really an antidote to the world of tech. There's something that I have come to through the classic stuff of, you know, mindfulness meditation and just hiking and being in nature, looking at trees and finding that beautiful.
You realize that there's a lot more psychological sufficiency in the act of just existing in the world, you know, controlling your own attention, letting the world just as it is, interest you and surprise you. And there's a problem, which is that, you know, no one's making money when you do that. And so this is kind of a macro version of what we were talking about earlier in our conversation about, you know, all models are wrong. And in some ways, the danger is that the models can reshape reality to become right.
In this case, we are creating these built environments in which there is just frankly not a lot of visual novelty because people live in an apartment with, you know, they can only view the building right next to them or there's not any trees near them or whatever it is. And so in order to get that like very primitive level of just visual surprise, you know, you didn't have to check your Twitter feed or whatever it might be. And that puts you into this world of status competition because that's how they get you to engage and create the content that other people consume. But if you just walked to your neighborhood park, you know, the park doesn't require anything from you.
And so I think it is worth bearing in mind. Like I'm at some level skeptical to the idea that, you know, everything we do is about this kind of positional good, this kind of status competition. When I'm walking through a park, I don't think this is really cool because other people aren't in this park. I mean, you appreciate the absence of other people for not kind of getting the way of your experience of nature, but you don't think about it as like, this is cool because it is scarce or I am the victor of this competition to be in this park, whatever it is.
But I get as much from that as I get from Instagram, you know, like that's funny thing. I guess that's actually a good place to bring it to an end here. So let's go back to analog for a minute. What are three books you'd recommend to the audience?
Yeah. So thinking about both what's coming down the highway and also kind of how do we think about human motivation and human desire. So the first book that comes to my mind is by Julie Shaw and Laura Major, Julie and I were high school classmates and she's now an MIT robot assistant who works on aerospace manufacturing and all sorts of things. Their book is called What to Expect When You're Expecting Robots.
I think it's a really interesting and persuasive look at kind of the next decade ish in terms of human robot interaction. I'm also thinking about a book by James Carson, who was a professor of religion at NYU called finite and infinite games. And there's this wonderful backstory to this book where he is a professor of religion who attended a game theory conference in the eighties and then writes this book, which is religion meets game theory meets like Vickin Stein's Tractatus. It's this very weird, very unique book that is all about what are people really trying to do?
And there are certain things that we do to achieve like a specific end that we can envision and advance other things that we do in this more sort of horizon open ended way to kind of surprise ourselves or prolong an experience. And I think it's a useful way of actually thinking about living one's life. It also maps to problems in AI in a very interesting way. The third one kind of coming back to what we were saying about the pleasure of being in the neighborhood park.
I'm thinking about a book by my friend Jenny Odell called How to Do Nothing. Resisting the attention economy. And it's in one level, you know, a love letter to her neighborhood park and at another level, you know, an invitation to think about a world in which most of our activity is directed at some kind of objective. And again, I think they're surprisingly relevant residences here with AI, you know, you can't make an AI system without an explicit objective function that it's trying to maximize.
What does it mean to quote unquote do nothing? And there's something I think powerful about that. Again, both for thinking about what intelligent machines might be like, but also in terms of thinking about these deep questions of human motivation, what makes life enjoyable? Brian Christian, your book is the alignment problem.
It is fantastic. I highly recommend it. Thank you very much. It's been pleasure.
Thank you. There's a collection of New York Times opinion is produced by Jeff Gelt, or J. Karma and Annie Galvin backtracking by Michelle Harris, original music by Isaac Jones and mixing by Jeff Gelt.