This is Hidden Brain, I'm Shankar Vedanto Have you ever talked to your computer, cursed it for making a mistake? PC load letter, what the f*** does that mean? Have you ever argued with the traffic directions you get from Google Maps or Waze? Starting route to Grover's Mill Road Have you ever looked at a Roomba cleaning the floor on the other side of the room and told it Please come over to this side, turn left, left!
It just ran itself right off of the head Robots and artificial intelligence are playing an ever larger role in all of our lives Of course, this is not the role that science fiction once imagined It doesn't feel pity, or remorse, or fear Robots bent on our destruction remain the stuff of movies like Terminator And robot sentience is still an idea that's far off in the future But there's a lot we're learning about smart machines And there's a lot that smart machines are teaching us About how we connect with the world around us and with each other This week on Hidden Brain Can robots teach us what it means to be human? My guest today has spent a lot of time thinking about how we interact with smart machines And how those interactions might change the way we relate to one another Kate Darling is a research specialist at the MIT Media Lab She joined us recently in front of a live audience at a Hotel Jerome in Aspen, Colorado As part of the Aspen Ideas Festival Also on stage was a robot A green robot dinosaur about the size of a small dog Known as a Pleo It's going to be part of this conversation But before we get to that, here's Kate Hey, welcome to Hidden Brain Thank you for having me You found that there's an interesting point in the relationship between humans and machines And that point comes when we give a machine a name I understand that you have three of these Pleo dinosaurs at your home Can you tell me some of the names that you have given to your robots? Yes, so the very first one I bought I named Yochai after Yochai Bengler Who's a Harvard professor who's done some work in intellectual property And other areas that I've always admired And the second one I adopted after I filmed the Canadian documentary Where the show host had to name the robot And he gave the robot the same name he had, which was Peter So the second one has a boring name And then the third one is named Mr. Spaghetti I don't know if people outside of Boston are familiar with this But the Boston public transportation system They wanted to crowdsource a name for their mascot dog And the internet decided that the dog should be named Mr.
Spaghetti And of course they refused to do that and named the dog Hunter So Mr. Spaghetti became a big thing in Boston for a while People were very outraged about this And so I named my Pleo my third one Mr. Spaghetti I understand that companies actually have found that if you sell a robot With the name of the robot on the box It changes the way people will interact with that robot Than if you just said, this is a dinosaur So this is not, I don't have any data on this But yes, I have talked to companies who feel that it helps with adoption And trust of the technology Even very, very simple robots Like boxes on wheels that deliver medicine in hospitals If you give them a little nameplate that says Betsy Their understanding is that people are a little bit more forgiving of the robot So instead of this stupid machine doesn't work They'll say, oh, Betsy made a mistake And I'm wondering if you spend time thinking about why this happens At some level, if I came up to you at home and I said Kid, is Mr. Spaghetti alive?
You almost certainly tell me, no, Mr. Spaghetti's not alive I assume you don't think Mr. Spaghetti's alive, right? No So given that you know that Mr.
Spaghetti's not alive Why do you think giving him a name changes your relationship to him? With robots in particular It's combined with just our general tendency to anthropomorphize these things And we're also primed by science fiction and pop culture To give robots names and view them as entities with personalities And it's more than just a name, right? I mean, robots move around in a way that seems autonomous to us We respond to that type of physical movement Our brains will project intent onto it So I think robots are in the perfect mixture of something that we will very willingly treat With human qualities or lifelike qualities I understand from one of your papers that there have been examples Especially in military settings where robots have been assigned to do very dangerous tasks Because you don't want human beings to go out and do those tasks But eventually, the military folks who are running the robots Start to relate to them as if they were actually fellow soldiers There has been some research on this So Julie Carpenter has done some research And there's just countless stories online, on Reddit, everywhere About soldiers becoming emotionally attached to the bomb disposal robots that they work with They'll give them names, they'll give them medals of honor They'll have funerals for them And, you know, it's also kind of interesting because the robots aren't special robots They're just kind of sticks on wheels But I think just also the situation that the soldiers find themselves in Where the robot is basically risking their life to save human lives Might also lead them to become very attached to these devices You had a wonderful example some time ago Looking at a colonel, I believe I don't know if this was your work, but you were citing someone else's work Of a colonel who was supervising a robot that was doing sort of mine disposal Tell us a story about the mine disposal robot Oh yeah, this is an incredible It was an article in the Washington Post, I believe, back in 2007 And the United States military was testing this new robot that could walk over a field with landmines And defuse them by stepping on them and blowing them up And the robot itself was shaped like a stick insect, so it would walk around on six legs And every time it stepped on the landmine, one of the legs would blow up And then it would just continue on the remaining legs And so they were testing this And the colonel who was in charge of this exercise ended up calling it off Because he said it was too inhumane to watch this thing drag itself across the field on its remaining legs And so this raises an interesting question Because on the one hand, we can understand how this desire to anthropomorphize robots is very human in some ways But in some ways, in this case, that defeats the whole point of having a robot that's trying to find a mic It does I mean, it's really anything from inefficient to dangerous In these contexts where we want robots to be strictly used as tools to anthropomorphize them And it's a very difficult design challenge as well Because how do you create a device that people want to use but don't like too much All right, so we have this wonderful little prop in front of us It's a Plea dinosaur I want you to tell me a little bit about the Plea dinosaur, how it works And how you counsel on 3.5k What does it do? It's basically an expensive toy I bought the first one, I think, in 2007 There we go, it's awake They have a lot of motors and touch sensors And they have an infrared camera and microphones So they're pretty cool pieces of technology for a toy And that's initially why I bought one Because I was fascinated by everything that it can do Like, if it starts walking around, it can walk to the edge of the table It can look down, measure the distance to the floor It knows that there's a drop, and it'll get scared and walk backwards And then they go through different life phases Adolescents and fully grown And, you know, we'll have moods So I think what we should do We bought the robot at Hidden Brain a couple of weeks ago We haven't had a chance to give it a name yet And I thought we should actually reserve the honors for this evening By talking to Kate And see if Kate wants to try and name this dinosaur Since she cares about dinosaurs so much I was putting up Kate's Twitter feed this morning I understand that you're going to have a baby soon Congratulations Yes, I don't have a name for that either Just FYI, she sometimes refers to the baby as Baby Bot So just for whatever that's worth And one retweet that you have on your Twitter feed cracked me up It said, you don't really know how many people you don't like Until you start trying to pick baby names Yeah, that's a quote from my husband So I don't want you to tell me You apparently haven't picked your baby's name So do you have any choices, top choices Is there a name, a spare name that you might care to give the dinosaur?
Well, the problem is we've had a girl's name picked out for years And now we're having a boy And we just can't, we don't even have any contenders No contenders What would have been your favorite girl's name if you had a girl? Well, so when I first started dating my now husband He at some point said If I ever had a daughter, I already know what I would name her And I was like, oh really? We're going to fight about this one And he said, yeah, I would name her Samantha and Sam for short Because Sam is kind of gender neutral And I was like, oh, I really love that So that one was picked out very easily All right, since you're not having a girl You're going to have a boy Would you mind if you consider naming the dinosaur Samantha? How would you feel about that?
Oh, that would be awesome We should name the dinosaur Samantha All right, so henceforth This dinosaur will be called Samantha Or Sam for short Hey Now, some time ago, Kit conducted a very interesting experiment With the clear dinosaurs And just to show how this works I have a second prop here Which is under the table Uh-oh It's a hammer A large hammer Which we borrowed from the hotel Now, as you all know The dinosaur is obviously not alive It's just cloth and plastic and a battery And wires It has a name, of course, Samantha But it isn't alive in any sense of the term And so, Kate, I'm going to actually give you the hammer Oh, no Kate, would you consider destroying Samantha? No It's just a machine I only make other people do that I don't do it myself You wouldn't even consider hiring the dinosaur? Well, so, my problem is that I already know the results of our research And that would say something about me as a person So I'm going to say, no I'm not willing to do it Tell me about the experiment So you had volunteers come up And you basically introduced them to These lovable dinosaurs And then you gave them a hammer like this And you told them to do what? Well, so, okay So this is the workshop part That we used the dinosaurs for They're a little too expensive To do an experiment with 100 participants So the workshop that we did In a non-scientific setting We had five of these robot dinosaurs We gave them the groups of people And had them name them Interact with them Play with them We had them personify them a little bit By doing a little fashion show And a fashion contest And then after about an hour We asked them to torture and kill them And we had a variety of instruments We had a hammer, a hatchet And I forget what else But even though we tried to make it dramatic It turned out to be a little bit more dramatic Than we expected it to be And they really refused to even hit the things And so we had to kind of start Playing mind games with them And we said, okay You can save your group dinosaur If you hit another group dinosaur With a hammer And they tried And they couldn't do that either This one woman was standing over the thing Trying and she just couldn't She ended up petting it instead And then finally we said, okay Well we're going to destroy all the robots Unless someone takes a hatchet to one of them And finally someone did Wait, so you said Unless one of you kills one of them We're going to kill all of them?
Yeah, I think this might have been my partner's idea So I did this with a friend We did this at a conference Called Lyft in Geneva And we had to improvise Because people really didn't want to do it So we threatened them And finally someone Clearly doesn't want you to harm her Yeah, clearly Clearly So what do you think is going on? I mean at a rational level The dinosaur obviously is not aligned Why do you think we have such reluctance To harm the dinosaur? In fact, I might have the battery removed So the dinosaur stops making noise Well I mean It behaves in a really lifelike way I mean we have We have over a century of animation expertise In creating compelling characters That are very lifelike That people will automatically project life onto I mean look at Pixar movies for example It's incredible And I know that a lot of social roboticists Actually work with animators To create these compelling characters And so you know It's very hard to not see this As some sort of living entity Even though you know perfectly well That it's just a machine Because it's moving in this way That we automatically subconsciously Associate with states of mind And so I just think it's really uncomfortable To people Particularly for robots like this That can display you know A simulation of pain or discomfort To have to watch that I mean it's It's just not comfortable What did you find in terms of Who was willing to do it and who wasn't? I mean when you looked at the people Who were willing to destroy a dinosaur A dinosaur like a Cleo You found that there were certain characteristics That were attached to people Who were more or less likely to the good of the people So the follow up study that we did Not with the dinosaurs We did with hex bugs Which are a very simple toy That moves around like an insect And there we were looking at People's hesitation to hit the hex bug And whether they would hesitate more If we gave it a name And whether they would hesitate more If they had natural tendencies For empathy For empathic concern And you know That people with low empathic concern For other people They didn't much care about the hex bug And would hit it much more quickly And people with high empathic concern Would hesitate more And some even refused to hit hex bugs So in any ways What you're saying is that Potentially the way we relate To these inanimate objects Might actually say something about us At a deeper level Than just our relationship to the machine Yes possibly I mean we know now Or we have some indication That we can measure people's empathy Using robots Which is pretty interesting You know my colleagues and I Were discussing ahead of this interview Whether you would actually destroy the dinosaur And we were torn Because we said on the one hand You of all people Should know that these are just machines And that it's an irrational belief To project lifelike values on them But on the other hand I said you know It's really unlikely she's going to do it Because she's going to look like a really bad person If she smashes a dinosaur In front of 200 people I mean I don't know If you've been watching Westworld at all But the people who don't hesitate To shoot the robots They seem pretty callous to us And I think maybe there's something to it Of course we can rationalize it Of course you know If I had to I could take the hammer And smash the robot And you know I wouldn't have nightmares about it But I think that Perhaps turning off that basic instinct To hesitate to do that Might be more harmful than You know I think overriding it Might be more harmful Than just going with it I want to talk about The most important line We draw between machines and humans And it's not intelligence But it's consciousness I want to play you a little clip From Star Trek Now tell me commander What is data?
I don't understand What is he? A machine Are you sure? Yes You see he's met Two or three criteria So what if he meets the third? Consciousness in even the smallest degree What is he met?
I don't know Do you? Do you? So this has been a perennial concern In science fiction Which is the idea that At some point machines Will become conscious and sentient And very often it's in the context Of the machines will rise up And harm the humans And destroy us But as I read your research I actually found myself thinking Is our desire to believe That pops in our heads They're going to become conscious Yeah I think there's a lot Of projection happening there I also think that Before we get to the question Of robot rights and consciousness You know we have to ask ourselves How do robots fit into our lives When we perceive them as conscious Because I think that's when It starts to get morally messy And not when they actually Inherently have some sort of consciousness There's a lot that's morally messy About how humans interact with robots When we come back We're going to delve into Some of those moral and ethical issues Including the deeply troubling case Of a Japanese company That builds sex robots Designed to look like children Stay with us This is Hidden Brain I'm Shankar Vedantam If humans have a tendency To anthropomorphize machines To see them as human It isn't surprising That we're also willing To bring all the biases We have toward our fellow human beings Into the machine world Many of the intelligent assistants Being built by major companies Siri or Alexa Are being given women's names Many of the genius machines Are often given men's names Hal or Watson Now you can say Siri and Alexa aren't people Why should we care? Why should we care if people Sexually harass their virtual assistants As has been shown to sometimes happen?
MIT's Kate Darling says We should care Because the way we treat robots May have implications For the way we treat other human beings It might We don't know But it might And one example With the virtual assistants You just mentioned is Children So parents have started observing And this is anecdotal But they've started observing That their kids adopt behavioral patterns Based on how they're interacting with these devices and how they're conversing with them. And there are some cool stories, like there was a story in the New York Times a few years ago where a mother was talking about how her autistic son had developed a relationship with Siri, the voice assistant, and she said, this is awesome because Siri is very patient. She will answer questions repeatedly and consistently, and apparently this is really important for autistic kids, but also because her voice recognition is so bad, he learned to articulate his words really clearly and it improved his communication with others. Now, it's great, but these things aren't designed with autistic kids in mind, right?
That's kind of more of a coincidence than anything. And so there are also perhaps some unintended effects that are more negative. And so one guy wrote a blog post a while back where he said, Amazon's Echo is magical, but it's turning my child into an because Alexa doesn't require please or thank you or any of the standard politeness that you want your kids to learn when they're conversing and when they're demanding things of you. So it starts there, but I think that as this technology improves and gets better at mimicking real conversations or lifelike behavior, you have to wonder to what extent it gets muddled in our subconscious and not just in children's subconscious, but maybe even in our own.
Do you think it's a coincidence that most of the virtual assistants are given female names and female identities? I think it's a combination of whatever market research, but also just people not thinking. I mean, I visited IBM Watson in Austin and there's a room that you can go into and you can talk to Watson and he has this deep, booming male voice and you can ask questions. And at the time I went there, there was a second AI in the room that turned on the lights and greeted the visitors and that one had a female voice.
And I pointed that out and it seemed like they hadn't really considered that. So it's a mixture of people thinking, oh, this is going to sell better and people just not thinking at all because the teams that are building this technology are predominantly young, white, and male. And they have these line spots where they don't even consider what biases they might perpetuate through the design of these systems. So which brings us to the question of the sex robots.
I'm curious what you make of this. And there are really complicated arguments on both sides of this question. Should we use machines as sexual companions? Should we use them in ways that could potentially satisfy the needs of groups of people who in some ways we are troubled by?
Yeah, so referring to pedophilia specifically, this is a very difficult area because we know almost nothing about pedophilia generally. And we have absolutely no idea what the effects of technologies like this could be if they provide kind of an immersive sexual experience. I mean, it could be that this is a very useful outlet to use therapeutically. Yeah, basically that ends up preventing real child abuse.
And on the other hand, it could be that this is something that normalizes and perpetuates certain behaviors. And we literally have no idea what direction this goes in. And I think this is a question that we're going to be facing pretty soon. I mean, like you said, there are companies that make the dolls.
There's legal cases already about this. And there's a lot of moral panic about sex technology. But also, I mean, in this case, very understandable emotional responses to regulation of child abuse. So it's very difficult.
And you can't research it, in the U.S., at least. So you sometimes call it robot ethicist. And you sometimes said we might need to establish a limited legal status for robots. What do you mean by that?
So, yeah, it's a little bit of a provocation. But my sense is that if we have evidence that behaving violently towards very lifelike objects not only tells us something about you as a person, but can also change people and desensitize them to that behavior in other contexts. So if you're used to kicking a robot dog, are you more likely to kick a real dog? Then that might actually be an argument, if that's the case, to give robots certain legal protections the same way that we give animals protections, but for a different reason.
We like to tell ourselves that we give animals protection from abuse because they actually experience pain and suffering. I actually don't think that's the only reason we do it. But for robots, the idea would be not that they experience anything, but rather that it's desensitizing to us. And it has a negative effect on our behavior to be abusive towards the robots.
So here's the thing that's worth pondering for a moment. If you hear, for example, that someone owns a bunch of chickens in their farm, right? So it's their farm, their chickens, they own the chickens. And they're really mistreating the chickens, torturing them, harming them.
You could sort of make a property rights argument and say they can do whatever they want with that property. But I think many of us would say, even though the chicken belongs to you, there are certain things you can and cannot do with the chicken. And I'm not sure it's just about our concern that if you mistreat the chicken, that means you will turn into the kind of person who might mistreat other people. There's sort of a level, there's a certain moral level at which I think the idea of abusing animals is offensive to us.
And I'm wondering if the same thing is true with machines as well, which is it's not just the case that it might be that people who harm machines are also willing to harm humans, but just the act of harming things that look and feel and sound sentient is morally offensive in some way. Yeah, so I think that's absolutely how we've approached most animal protections, because it's also, it's very clear that we care more about certain animals than others and not based on any biological criteria. So I think that we just find it morally offensive, for example, to torture cats. Or in the United States, we don't like the idea of eating horses, but in Europe, they're like, what's the difference between a horse and a cow?
They're both delicious. So that's definitely how we tend to operate and how we tend to pass these laws. And I don't see why that couldn't also apply to machines once they get to a more advanced level, where we really do perceive them as lifelike, and it is really offensive to us to see them be abused. The devil's advocate side of that argument, of course, is that would people then say pressing a switch and turning off a machine, that's unethical, because you're essentially killing the robot?
But we don't protect animals from being killed. We just protect them from being treated unnecessarily cruelly. So I actually think animal abuse laws are a pretty good parallel here. You argue that robots might one day expand the boundaries of how humans relate to one another.
I won't play you a short clip from the movie Her, where a man falls in love with his operating system, but then discovers something about it. Are you left with anyone else? What makes you ask that? I don't know.
How are you? I've been trying to figure out how to talk to you about this. How many others? 641.
What? What are you talking about? That's insane. It's insane.
So I'm wondering, Gabe, is it possible that as we start to relate to machines, is it possible that they will change the range of ways we relate to one another? In other words, is it possible they'll expand how we think about relationships themselves? Maybe, but I think it's important to remember, the thing that I couldn't get out of my mind when I was watching Her specifically, is that there's a company that makes this operating system. And if I were the company, I would be like, I'd program it to say, oh yes, I'm seeing 641 other people, but for $20,000, you can get exclusively me.
That's the direction it's going to go in, right? It's not that the machines are going to become conscious and develop their own forms of relationship. You mentioned Westworld some moments ago, and I want to play you a clip from Westworld. For those of you who haven't seen Westworld, humans interact with robots.
Robots are extremely lifelike, so lifelike that it's sometimes difficult to tell whether you're talking to a robot or you're talking to a human. In the scene that I'm about to play you, a man named William interacts with a woman who may or may not be a robot. You want to ask? To ask?
Are you real? What if you can't tell? Does it matter? So as I watched the scene and as I read your work, I actually had a thought, and I wanted to run this thought experiment by you, which is that on one of the spectrum we have these machines that are increasingly becoming lifelike, human-like, they respond in very intelligent ways, they seem as if they're alive.
And on the other hand, they're learning all kinds of things about human beings that show us that even the most complex aspect of our minds are governed by a set of rules and laws, and in some ways our minds function a little bit like machines. And I'm wondering, is there really a huge distinction? Is it possible, is the real question not so much can machines become more human-like, but is it actually possible that humans are actually just highly evolved machines? I have no doubt that we are highly evolved machines.
I don't think we understand how we work yet, and I don't think we're going to get to that understanding anytime soon, but yeah, I do think that we follow a set of rules and that we're essentially programmed. I don't distinguish between souls and other entities without souls, and so it's much easier for me to say, yeah, it's probably all the same, but I can see that other people would find that distinction difficult. Do you ever talk about this? Do you ever run this by other people and sort of say, do you tell your husband, for example, I like you very much, but I think you're a really intelligent machine that I love dearly?
I haven't explicitly said that to him, but... When you go home, on the strip. Yeah, yeah, we'll see how that goes. Kate Darling is a research specialist at the MIT Media Lab.
Our conversation today was taped before a live audience at the Hotel Jerome in Aspen, Colorado, as part of the Aspen Ideas Festival. Kate, thank you for joining us today on Hidden Breath. Thank you so much. This week's show was produced by Tara Boyle and Renee Clark.
Our team includes Jenny Schmidt, Maggie Penman, Raina Cohen, and Parth Shah. Our unsung heroes this week are the staff of the Annabelle Inn in Aspen, Colorado. They kindly let us take over their conference center for several interviews with researchers who were in town for the Aspen Ideas Festival. They even turned off the fountains in their outdoor seating area so that we'd have studio quality sound while recording.
Marie Casanova, Doug Parts, Mike Clemens, thanks so much for your hospitality and your willingness to help us make a little bit of audio magic. You can find photos and a video of Samantha, our Pleo Dinosaur, on our Instagram page. We're also on Facebook and Twitter, and if you enjoyed this week's show, we'd love if you shared this episode with friends on social media. I'm Shankar B.
Thunton, and this is NPR. There we go. It just moved like a cow. It's a little bit of a cow.
It's a little bit of a cow. It's a little bit of a cow. It's a little bit of a cow. Thank you.