Skip to content
Radio Replay: I, Robot episode artwork

Jan 12, 2018 48 min

Radio Replay: I, Robot

from Hidden Brain Hosted by Hidden Brain Media

Do you ever catch yourself yelling at your Alexa? Or typing questions into Google that you wouldn't dare ask aloud? On this episode, our changing relationship with technology and what big data knows about our deepest, darkest secrets.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Radio Replay: I, Robot

0:00 48:56
of matches

Transcript Auto-generated, may contain errors

Quick note before we get started, this episode includes a racial epithet and discussions about pornography. This is Hidden Brain, I'm Shankar Vedanta. Maps are good representations not just of the world we live in, but of how we think about the world we live in. Over the centuries, our maps have emphasized the places we find important.

They show the limits of our knowledge and the scope of our ambitions. 700 years ago, Europeans were completely unaware of the existence of North America. Fast forward to the 1970s. Scientists can tell you in detail what the surface of the moon is like.

Today we're charting out maps of a different sort. Maps of our minds. Maps of our minds. Maps of our minds.

There isn't one cartographer designing these modern maps. We all are. And the maps are constantly changing. We start today's show with a personal question.

Have you ever googled something that you would never dream of saying out loud to another human being? When we have a question about something embarrassing or deeply personal, many of us don't turn to a parent or to a friend, but to our computers. Because there's just some things you just can't ask a real person in real life and you need to ask Google. Because it's completely anonymous and there are no judgments attached.

Google knows everything. I agree to that. Every time we type into a search box, we reveal something about ourselves. As millions of us look for answers to questions or things to buy or places to meet friends, our searches produce a map of our collective hopes, fears and desires.

My guest today is Seth Stevens-Dividowitz. He used to be a data scientist at Google and he's the author of the book, Everybody Lies. Big Data, New Data and what the internet can tell us about who we really are. Seth, welcome to Hidden Brain.

Oh, thanks so much for having me, Shankar. So Seth, we all know that Google handles billions of searches every day, but one of the insights you've had is that the reason Google knows a lot about us is not just because of the volume of search terms. But because people turn to Google as they might turn to a friend or a confidant. That's exactly right.

I think there's something very comforting about that little white box that people feel very comfortable telling things that they may not tell anybody else about their sexual interests, their health problems, their insecurities, and using this anonymous area of data, we can learn a lot more about people than we've really ever known. And one of the ways we can learn a lot more about people is through these very strange correlations. You find, for example, there's a relationship between the unemployment rate and the kinds of searches people make online. Yeah, I was looking at what searches correlate most with the unemployment rate and I was expecting something like new jobs or...

Unemployment benefits, but during the time period, I looked at the single search that was most highly correlated with the unemployment rate with a pornography site. And you can imagine that if a lot of people are out of work, they have nothing to do during the day, they may be more likely to look at porn sites and other search that was high on the list was solitaire. So again, when people are out of work, they're out of work, they do leisure activities, and potentially this measure of how much leisure there is on the internet may help us know how many people are out of work on a given day. And of course, this sort of helps us reconsider what we think of as data.

So when we think about the unemployment rate, as you say, our normal approach is to say how many people are selling jobs, let's track down all the jobs. This is coming at the question entirely differently. Yeah, I think the traditional way to collect data was to send a survey out to people and have them answer questions, checkboxes. There are lots of problems with this approach.

Many people don't answer surveys and many people lie to surveys. So the new era of data is kind of looking through all the clues that we leave, many of them not as part of questions or as part of surveys, but just clues we leave as we go through our lives. One of the important differences between mining this kind of data and the responses we get on surveys has to do with how people report their sexual orientation. I understand that the kind of queries that you see on Google might reveal something quite different than if you ask people if they're gay.

That's right. If you ask people in surveys today in the United States about two and a half or three percent of men say that they're primarily attracted to men. And this number is far higher in certain states where tolerance to homosexuality is greater. So there are a lot more gay men according to surveys in California than in Mississippi.

But if you look at search data for gay male pornography, it's a tiny bit higher in California, but not that much higher. And overall, about five percent of male pornography search is for gay porn. So almost twice as high as the numbers you get in surveys. Your research has important implications for a topic that we've looked at a lot on Hidden Brain, the topic of implicit bias.

People aren't always aware of the biases they hold. And so scientists have had to find clever ways to unearth these biases. You think that Google searches can reveal some forms of implicit bias? That's right.

So one I look at is the questions that parents have about their children. If you ask many parents today, they would say that they treat their sons and daughters equally, that they're equally excited about their intellectual potential, equally concerned about maybe their weight problems. But if you aggregate everybody's Google searches, you see large differences in gender that when parents in the United States ask questions starting, is my son, they're much more likely to use words such as gifted or a genius than they would in a search starting, is my daughter? When parents in the United States search, is my daughter, they're much more likely to complete it with, is my daughter overweight or is my daughter ugly?

So parents are much more excited about the intellectual potential of their sons and much more concerned about the physical appearance of their daughters. Before I get to the next question set, I just want to give a warning to our listeners. This next section is going to involve a discussion regarding the N word. So I'll report that in some states after Barack Obama was first elected president, there were more Google searches for a certain racist term than searches for black president.

I think there is a disturbing element to some of this search data where in the United States today, many people, and maybe this is a good thing, don't feel comfortable sharing that they have racist thoughts or racist feelings. But on Google, they do make these searches in strikingly high frequency. I need to use sort of a language to this. The measure is the percent of Google searches that include the word.

And these searches are predominantly searches looking for jokes, mocking African Americans. I should clarify, this is not searches for rap lyrics, which tend to use the word ending in A. But if you look at the racist search volumes, I think if you had asked me based on everything I had read about racism in the United States, I would have thought that racism in the United States predominantly concentrated in the South. That really the big divide in the United States when it comes to racism is South versus North.

But the Google data reveals that's not really the case, that racism is actually very, very high in many places in the North, places like Western Pennsylvania, Eastern Ohio, or industrial Michigan, or rural Illinois, or upstate New York. The real divide these days when it comes to racism is not North versus South, it's East versus West. There's much higher racism, East of the Mississippi than West of the Mississippi. So besides just saying, you know, we know that there are these patterns of racist searches in different parts of the country, you're actually saying you can do more than that.

You can actually predict how different parts of the country might vote in a presidential election based on the kind of Google searches you see in different parts of the country. Well, the first thing I found is that there was a large correlation between racist search volume and parts of the country where Obama did worse than other Democratic candidates had done. So Barack Obama was the first major party general election nominee who was African American, and you see a clear relationship that Obama lost large numbers of votes in parts of the country where there are high racist search volumes and other researchers have found such as Nate Silver at 538 and Nate Cohn at the New York Times, that there was a large correlation between racist search volumes and support for Donald Trump in the Republican Party that parts of the country that made racist searches in high numbers were much more likely to support Donald Trump. And this relationship was much stronger than really any other variable that they tested.

I'm wondering how you try and understand that kind of information. It's hard not to listen to what you're saying and draw sort of what seems to be a superficial conclusion, which is that racist people vote for Donald Trump. I'm not sure. Is that what you're saying?

That's one of those things where it sounds so offensive to say it that I think everyone tiptoes around the line. I will say that the data does show a strong correlation between racist searches and support for Donald Trump that is hard to explain with any other explanation. I'm not saying that everybody who supported Donald Trump is racist by any stress imagination. There are plenty of people who support Donald Trump without this racist tendency.

But a significant fraction of his supporters are motivated by racial animus. Seth Stevens, the former Google Data Scientist and the author of Everybody Lies, Big Data, New Data, and what the internet can tell us about who we really are. You spend a lot of time in the book talking about sex. It turns out to be an area where marketers and companies know that what we say about ourselves is nowhere close to the truth.

Most people report being not interested in pornography, but the website Pornhub reports that in 2015 alone, viewers watched 2.5 billion hours of porn, which is apparently longer than the entire amount of time that humans have been on Earth. What does this say about us? The fact that we either have very little insight about ourselves or we're actually lying through our teeth. Yeah, I'd say we're probably lying through our teeth.

I do talk a lot about sex in this book. One thing I like to say is that Big Data has so powerfully turned me into a sex expert because it wasn't a natural area of expertise for me. I do talk a lot about sexuality and I think you do learn a lot about people that's very, very different from what they say and kind of the weirdness at the heart of the human psyche that doesn't really reveal itself in everyday life or at lunch tables, but does reveal itself at 2am on Pornhub. My family was going back and forth.

My sister was saying that I need a crazy girl because I'm crazy. My brother was saying that my sister was crazy that I need a normal girl to balance me out and my mom was screaming at my brother and sister that I'm not crazy and my dad was then screaming at my mom that of course is crazy. So it's kind of a classic Stephen Sividowitz family Thanksgiving where everyone just yelling at each other for being crazy and not really getting any progress and learning about what I need in my love life. My soft-spoken 88-year-old grandma starts to speak and everyone went quiet and she explains to me that I need a nice girl, not too pretty, very smart, good with people, social, see with two things, sense of humor because you have a good sense of humor.

And I describe why it was her advice so much better than everybody else's. I think one of the reasons that she's big data, right? So grandmas and grandpas throughout history have had access to more data points than anybody else and they've been able to correlate larger patterns than anybody else has because they've been around longer. And that's why they've been such an important source of wisdom historically.

The problem of course, as you also point out, is that it's very hard to disentangle your personal experiences from what actually happens in the world. And in your grandmother's case, she actually had a very specific piece of relationship advice about the kind of person you should want. And some of that might not actually be backed up by the empirical evidence. Yeah, well, my grandma has told me on multiple occasions that it's important to have a common set of friends and a partner.

So she lived in a small apartment in Queens, New York with my grandfather and every evening they'd go outside and gossip with their neighbors. And she thought that was a big part in why their relationship worked. But actually recently, computer scientists have analyzed data from Facebook and they can actually look when people are in relationships and when they're out of relationships and try to predict what factors or relationships make it more likely to last. One of the things they tested was having a common group of friends.

Some partners on Facebook share pretty much the same friend group and some people have totally isolated friend groups. And they found, contrary to my grandmother's advice, that having a separate social circle is actually a positive predictor of a relationship lasting. And so of course, the risk of trusting the individual is that the individual's intuition about what work for his life might not work for everyone else. That's right.

I think we tend to get biased by our own situation. Data scientists have a phrase called waiting data. Some data points get extra weight in our models and our intuition gives too much weight to our own experience. And we tend to assume that what worked for us will work for others as well.

And that's frequently not the case. Many companies know that we don't really understand ourselves. When we come back, we look at how companies are using big data to predict what we're going to do before we know it ourselves. We'll also ask if sites like Google can use data to forecast whether you're going to get a serious illness, should they give you that information?

Stay with us. Starting a business is tough. Shopify makes it easier. Build a beautiful store, write with AI and manage marketing, inventory and shipping all in one place.

Join millions who trust Shopify to launch and grow their business effortlessly. If you're ready to sell, you're ready for Shopify. Turn your big business idea into sign up for your one euro per month trial at Shopify.ie. Go to Shopify.ie.

It's time to see what you can accomplish with Shopify by your side. At Vodafone, you get more. Like more than more. Get the brand new Samsung Galaxy S26 Ultra for zero euro plus get more peace of mind.

With an extra 12 month warranty, when you switch to our unlimited bill pay mobile plan. Now that's a whole lot more. Vodafone. Together we can.

Subject to availability eligibility and switching to our 24 month 65 euro per month unlimited bill pay plan by 30th of June, 26th. Each April the price of your plan will increase by 3 or 50 per month for full-term C Vodafone.ie for slash terms. The bank holiday is finally here. Enjoy summer savers at Central with great offers.

Like Central Fresh our Stripper on stage 2 pack only 8 euro offer in Sunday. Selector Pringles 165 gram 175 each plus Diet Coke and Coke Zero 12 gampage. Only 6 euro each plus deposit. Bank holiday value sorted.

Sentra. Live every day. This is Hidden Brain. I'm Shankar Vedantam.

We're speaking today with former Google data scientist Seth Stevens Davidowitz about the research in his book, Everybody Lies. Big Data, New Data and what the internet can tell us about who we really are. Netflix used to ask users what kind of movies they wanted to watch. Seth says eventually the company realized that asking this kind of question was a complete waste of time.

Yeah, initially Netflix would ask people what they want to view in the future so they could queue up with the movies that they said. And if you ask people what are you going to want to watch tomorrow or this weekend, people are very aspirational. They want to watch documentaries about World War II or avant-garde French films. But then when Saturday or Sunday comes around, they want to watch the same lowbrow comedies that they've always watched.

So Netflix realized they had to just ignore what people told them and use their algorithms to figure out what they actually want to watch. So one of the things that's intriguing about what you just said is that I don't think it's actually the case that people were lying to Netflix when they said they wanted to watch the avant-garde film. They actually genuinely probably aspired to do that. It might actually be that big data understands people better than they understand themselves.

Yeah, probably even more common than lying to other people is lying to ourselves, particularly when we're trying to predict what we're going to do in two or three days. We tend to assume that we're going to go to the gym more than we go to the gym or eat better than we actually will eat or watch more intellectual stuff than we actually will watch. So the algorithms can correct for this over-optimism that we all tend to share. When you look at a company like Facebook, which has access to these huge amounts of data about us and what we like and whom we like and our relationships, you have to wonder how the company is using this data in all kinds of different ways.

I remember Facebook got into some hot water a couple of years ago because they ran an experiment that seemed to be manipulating how people feel. And of course, there was a huge outcry about the experiment at the time. And since then, there hasn't been very much reported about what Facebook is doing, but I suspect it might just be because Facebook is no longer telling us what it's doing, but it's still doing it anyway. Every major tech company now runs lots and lots of what are called AB tests, which are little experiments where you put people into two different groups, a treatment and control group, and you show one group, one version of your site and the other group, another version of the site, and you see which version gets the most clicks or the most views.

This is really exploded in the tech industry. There are many, many instances where companies are now using big data against us. Banks and other financial institutions are using clues from big data to decide who shouldn't get a loan. I think it's an area of a big concern, so I talk about a study in the book where they cite a peer-to-peer lending site and they cite the text that people used in their request for loans, and you can figure out just from what people say in their loans how likely they are to pay back.

And there are some strange correlations. For example, if you mentioned the word God, you're 2.2 times less likely to pay back, 2.2 times more likely to default. And this does get eerie. Are you really supposed to be penalized if you mention God in a loan application?

That would seem to be really wrong, even evil, to penalize somebody for a religious preference. Basically, everything is correlated with everything. So just about anything anybody does is going to have some predictive power for other things they do. And the legal system is really not set up for a world in which companies potentially can mine correlations over just about everything anybody does in their life.

I was thinking about an ethical issue. I'm not sure if necessarily this is a legal issue, but you mentioned in the book that if someone is googling, I've been diagnosed with pancreatic cancer, what should I do? It's reasonable to assume that this person has been diagnosed with pancreatic cancer. But if you collect all the people who are googling what to do about their diagnosis with pancreatic cancer and then work backwards to see what they've been searching for in the weeks and months prior to their diagnosis, you can discover some pretty amazing things.

Yeah, this is a study that researchers use Microsoft Bing data. They looked at people who searched for just diagnosis of pancreatic cancer, and then similar people who never made such a search. And then they looked at all the health symptoms they had made in lead up to either a diagnosis or no diagnosis. And they found that there were very, very clear patterns of symptoms that were far more likely to suggest a future diagnosis.

Of pancreatic cancer. For example, they found that searching for indigestion and then abdominal pain was evidence of pancreatic cancer. While searching for just indigestion without abdominal pain, meant a person was much more unlikely to have pancreatic cancer. And that's a really, really subtle pattern in symptoms, right?

Like a time series of one symptom followed by another symptom is evidence of a potential disease. It really shows, I think, the power of this data where you can really tease out very subtle patterns in symptoms and figure out which ones are potentially threatening and which ones are benign. So here's the ethical question. Once you establish that there is this correlation, that you sort of say I have a universe of people who clearly have pancreatic cancer and work backwards through their search history, and I detect these patterns that no one had thought to look at before.

Let's say these particular kinds of search terms seem to be correlated with people who go on to have the diagnosis versus these search terms that do not go on to predict a diagnosis. So does a company like Microsoft now have an obligation to tell people who are googling for these combinations of search terms? Look, you might actually need to get checked out. You might actually need to go see a doctor.

Because of course, if you can be diagnosed with pancreatic cancer four weeks earlier, you have a much better chance of survival than if you have to wait for a month. I lean in the direction of yes, some people would not lean that direction. It could be a little creepy. If you go right below the body and I feel lucky, you may have pancreatic cancer.

It's not exactly the most friendly thing to see on a website. But personally, if I had some sort of symptom pattern that suggested I may have a disease and there was a chance of curing it, if I was told, I'd want to know that. It's just another example that really the ethical and legal framework that we've set up is not necessarily prepared for big data. Seth Stevens-Dividowitz is a former data scientist at Google and the author of the book, Everybody Lies, Big Data, New Data, and what the internet can tell us about who we really are.

Seth, thank you for joining me today on Hidden Brain. Thanks so much for having me, Shunker. Have you ever talked to your computer, cursed it for making a mistake? Please load letter.

Does that mean? Have you ever argued with the traffic directions you get from Google Maps or Waze? Starting route to Grover's Hill Road. Have you ever looked at a Roomba, cleaning the floor on the other side of the room and told it, please come over to this site?

Turn left. It just ran itself right off the edge. Robots and artificial intelligence are playing an ever larger role in all of our lives. Of course, this is not the role that science fiction once imagined.

It doesn't feel pity or remorse or fear. Robots bent on our destruction remain the stuff of movies like Terminator, and robots' sentience is still an idea that's far off in the future. But there's a lot we're learning about smart machines and there's a lot that smart machines are teaching us about how we connect with the world around us and with each other. My guest today has spent a lot of time thinking about how we interact with smart machines and how those interactions might change the way we relate to one another.

Kate Darling is a research specialist at the MIT Media Lab. She joined us recently in front of a live audience at the Hotel Jerome in Aspen, Colorado as part of the Aspen Ideas Festival. Also on stage was a robot, a green robot dinosaur about the size of a small dog known as a pliote. It's going to be part of this conversation, but before we get to that, here's Kate.

Kate, welcome to Hidden Brain. Thank you for having me. You found that there is an interesting point in the relationship between humans and machines, and that point comes when we give a machine a name. I understand that you have three of these pliote dinosaurs at your home.

Can you tell me some of the names that you have given to your robots? Yes. The very first one I bought, I named Yochai after Yochai Venkler, who's a Harvard professor who's done some work in intellectual property and other areas that I was admired. The second one I adopted after I filmed the Canadian documentary where the show host had to name the robot and he gave the robot the same name he had, which was Peter.

The second one has a boring name. The third one is Mr. Spaghetti. I don't know if people outside of Boston are familiar with this, but the Boston public transportation system, they wanted to crowdsource a name for their mascot dog.

The internet decided that the dog should be named Mr. Spaghetti. They refused to do that and named the dog hunter. Mr.

Spaghetti became a big thing in Boston for a while. People were outraged about this. I understand that companies have found that if you sell a robot with the name of the robot on the box, it changes the way people interact with that robot. This is a dinosaur.

I don't have any data on this. I have talked to companies who feel that it helps with adoption and trust of the technology. They are very simple robots like boxes on wheels that deliver medicine in hospitals. If you give them a little name plate that says Betsy, they are understanding that people are giving the robot.

Instead of this machine, they will say Betsy made a mistake. I'm wondering if you spent time thinking about why this happened. At some level, if I came up to you at home and said, is Mr. Spaghetti alive?

You would almost tell me no. I assume you don't think Mr. Spaghetti is alive. Given that you know that Mr.

Spaghetti is not alive, why do you think giving him a name changes your relationship to him? With robots in particular, it is combined with just our general tendency to anthropomorphize these things. We are also primed by science fiction and pop culture to give robots names and view them as entities with personalities. It is more than just the name.

Robots move around in a way that seems autonomous to us. We respond to that type of physical movement. Our brains will project intent onto it. I think robots are the perfect mixture of something that we will very willingly treat with human qualities or lifelike qualities.

We have this wonderful prop in front of us. It is a play-out dinosaur. I want you to tell me a little bit about the play-out dinosaur, how it works and how you count to one three of them. What is the dinosaur?

What does it do? It is basically an expensive toy. I bought the first one in 2007. They have a lot of motors and touch sensors and they have an infrared camera and microphones.

They are pretty cool pieces of technology for a toy. That is why I bought one because it was fascinated by everything that it can do. If it starts walking around, it can walk to the table. It can look down and measure the distance to the floor.

It knows there is a drop and it will get scared and walk backwards. Then they go through different life phases, adolescence and fully grown. It will have moods. I think what we should do is we bought the robot at Hidden Brain a couple of weeks ago.

We have not had a chance to give it a name yet. I thought we should reserve the honors for this evening by talking to Kate. She wants to try and name this dinosaur. She cares about dinosaurs so much.

I was looking up Kate's Twitter feed this morning. I understand you will have a baby soon. I don't know if you can't speak to that. I know you don't know how many people you don't like.

I don't want to tell you how many people you don't like. Do you have any choices? Is there a name or spare name you might care to give the dinosaur? The problem is we have a girl's name picked out for years and now we are having a boy.

We don't have any contenders. What would have been your favorite girl's name? If you had a girl. When I first started dating my now husband, he said if I had a daughter, I would know what I would name her.

I was like, oh, really? We are going to fight about this one. He said, yeah, I would name her Samantha and Sam for short because Sam is gender neutral. I was like, oh, I really love that.

That one was picked out very easily. Since you are not having a girl, you have a boy, would you mind if you considered naming the dinosaur Samantha? That would be awesome. We should name the dinosaur Samantha.

So henceforth, this dinosaur will be called Samantha or Santa. Now, sometime ago, Kate conducted an experiment with the dinosaur and I am sure this works. I have a second prop here which is under the table. It is a hammer.

A large hammer which we borrowed from the hotel. As you all know, the dinosaur is not alive. It is just cloth and plastic and a battery and wires. It has a name, of course, Samantha.

It isn't alive in any sense of the term. Kate, I will give you the hammer. I think we might have a little board underneath the table here. We are going to place the dinosaur on a board.

Kate, would you consider destroying Samantha? No. It is just a machine. I only make other people do that.

I don't do it myself. You would not consider harming the dinosaur? My problem is that I already know the results of our research and that would say something about me as a person. I am not willing to do it.

Kate Darling is a research specialist at the MIT Media Lab. When we come back, I last were about that research she references in which she asked volunteers to smash a robot dinosaur. Starting a business is Toph. Shopify makes it easier.

Build a beautiful store, write with AI and manage marketing, inventory and shipping all in one place. Join millions who trust Shopify to launch and grow their business effortlessly. If you are ready to sell, you are ready for Shopify. Turn your big business idea into sign up for your one euro per month trial at Shopify.ie.

Go to Shopify.ie. It is time to see what you can accomplish with Shopify by your side. At Vodafone, you get more. Like more than more.

Get the brand new Samsung Galaxy S26 Ultra for 0 euro. Plus, get more peace of mind with an extra 12 month warranty when you switch to our unlimited bill pay mobile plan. Now that's a whole lot more. Vodafone.

Together we can. Live every day. Welcome back to Hidden Brain. I'm Shankar Vethantam.

We are discussing our relationships with technology, specifically robots, with Kate Darling, a researcher from MIT. She joined us before a live audience at the Aspen Ideas Festival. A couple of years ago, Kate conducted an experiment that said a lot about how humans tend to respond to certain kinds of robots. Tell me about the experiments.

You had volunteers come up and you basically introduced them to these lovable dinosaurs and then you gave them a hammer like this and you told them to do what. Okay, so this was the workshop part that we used the dinosaurs for. They are a little too expensive to do an experiment with a hundred participants. So the workshop that we did in a non-scientific setting, we had five of these robot dinosaurs.

We gave them the groups of people and have them name them, interact with them, play with them. We had them personify them a little bit by doing a little fashion show with a fashion contest. And then after about an hour, we asked them to torture and kill them. And we had a variety of instruments.

We had a hammer, a hatchet, and I forget what else. But even though we tried to make it dramatic, it turned out to be a little bit more dramatic than we expected it to be. And they really refused to even hit the things. And they had to kind of start playing mind games with them.

We said, okay, you can save your group dinosaur if you hit another group dinosaur with a hammer. And they tried and they couldn't do that either. This one woman was standing over the thing trying and she just couldn't, she ended up petting it instead. And then finally we said, okay, well we are going to destroy all the robots unless someone takes a hatchet to one of them.

And finally someone did. Wait, so you said unless one of you kills one of them? We are going to kill all of them? Yeah, I think this might have been my partner's idea.

So I did this with a friend named Hana Skoslitz. We did this at conference called Lift and Geneva. And we had to improvise because people really didn't want to do it. We threatened them.

And finally someone did it. So now that T.A. doesn't want you to harm her? Yeah, clearly.

Clearly. So what do you think is going on? I mean at a rational level the dinosaur obviously is not an eye. Why do you think we have such an up-dense to harming the dinosaur?

In fact I might have the battery removed so the dinosaur stops making noise. Well, I mean it behaves in a really lifelike way. I mean we have over a century of animation expertise in creating compelling characters that are very lifelike that people will automatically project life onto you. I mean look at Pixar movies for example.

It's incredible. And I know that a lot of social roboticists actually work with animators to create these compelling characters. And so it's very hard to not see this as some sort of living entity even though perfectly well that it's just a machine because it's moving in this way that we automatically subconsciously associate with states of mind. And so I just think it's really uncomfortable to people, particularly for robots like this that can display a simulation of pain or discomfort to have to watch that.

I mean it's just not comfortable. What did you find in terms of who was willing to do it and who wasn't? I mean when you looked at the people who were willing to destroy a dinosaur, a dinosaur like a clayl, you found that there were certain characteristics that were attached to people who were more or less likely to do the deed. So the follow up study that we did, not with the dinosaurs we did with hex bugs which are a very simple toy that moves around like an insect.

And there we were looking at people's hesitation to hit the hex bug and whether they would hesitate more if we gave it a name and whether they would hesitate more if they had natural tendencies for empathy for empathic concern. And we found that people with low empathic concerns for other people they didn't much care about the hex bug and would hit it much more quickly. And people with high empathic concern would hesitate more and some even refused to hit the hex bugs. So in any way what you're saying is that potentially the way we relate to these inanimate objects might actually say something about us at a deeper level than just our relationship to the machine.

Yes, possibly. I mean we know now or we have some indication that we can measure people's empathy using robots which is pretty interesting. You know my colleagues and I were discussing ahead of this interview whether you would actually destroy the dinosaur. And we were torn because we said on the one hand you of all people should know that these are just machines and that it's an irrational belief to project life-like values on them.

But on the other hand I said it's really unlikely she's going to do it because she's going to look like a really bad person if she smashes the dinosaur in front of 200 people. I mean I don't know if you've been watching Westworld at all but the people who don't hesitate to shoot the robots they seem pretty callous to us. And I think maybe there's something to it. Of course we can rationalize it.

Of course if I had to I could take the hammer and smash the robot and I wouldn't have nightmares about it. But I think that perhaps turning off that basic instinct to hesitate to do that might be more harmful than over you know I think overriding it might be more harmful than just going with it. I want to talk about the most important line we draw between machines and humans and it's not intelligence but it's consciousness. I want to play your little clip from Star Trek.

Now tell me Commander what is data. What is he? A machine? Is he?

Are you sure? Yes. You see he's met two of your three criteria for centuries so why do you need to third consciousness and even the smallest degree? What is he then?

I don't know. Are you? So this has been a perennial concern in science fiction which is the idea that at some point machines will become conscious and sentient. And very often it's in the context of the machines will rise up and harm the humans and destroy us.

But as I read your research I actually found myself thinking is our desire to believe that the machines can become conscious. Actually just an extension of what we've been talking about in the last 20 minutes which is reprojects, sentience onto machines all the time. And so when we imagine what they're going to be like in the future the first thing that pops in our head is they're going to become conscious. Yeah I think there's a lot of projection happening there.

I also think that before we get to the question of robot rights and consciousness you know we have to ask ourselves how do robots fit into our lives when we perceive them as conscious because I think that's when it starts to get morally messy and not when they actually inherently have some sort of consciousness. If humans have a tendency to anthropomorphize machines to see them as human it isn't surprising that we're also willing to bring all the biases we have to word our fellow human beings into the machine world. Many of the intelligent assistants being built by major companies Siri or Alexa are being given women's names. Many of the genius machines are often given men's names Hal or Watson.

Now you can say Siri and Alexa aren't people. Why should we care? Why should we care? Why should we care if people sexually harass their virtual assistants as has been shown to sometimes happen?

MIT's kid Darling says we should care because the way we treat robots may have implications for the way we treat other human beings. It might. We don't know but it might. And one example with the virtual assistants you just mentioned is children.

So parents have started observing and this is anecdotal but they started observing that their kids adopt behavioral patterns based on how they're interacting with these devices and how they're conversing with them. There's cool stories like there was a story in the New York Times a few years ago where a mother was talking about how her autistic son had developed a relationship with Siri, the voice assistant. And she said this was awesome because Siri is very patient. She will answer questions repeatedly and consistently and apparently this is really important for autistic kids.

But also because her voice recognition is so bad he learned to articulate his words really clearly and it improved his communication with others. Now it's great but these things aren't designed with autistic kids in mind. That's kind of more of a coincidence than anything. And so there are also perhaps some unintended effects that are more negative.

And so one guy wrote a blog post a while back where he said Amazon's echo is magical but it's turning my child into an c*** because Alexa doesn't require please or thank you or any of the standard politeness that you want your kids to learn when they're conversing and when they're demanding things of you. So it starts there but I think that as this technology improves and gets better at mimicking real conversations or life like behavior you have to wonder to what extent it's muddled in our subconscious and not just in children's subconscious but maybe even our own. Do you think it's a coincidence that most of the virtual assistants are given female names and female identities? I think it's a combination of whatever market research but also just people not thinking.

I visited IBM Watson in Austin and there's a room that you can talk to Watson and he has this deep booming male voice and you can ask questions. And at the time I went there was a second AI in the room that turned on the lights and greeted the visitors and that one had a female voice and I pointed that out and it seemed like they hadn't really considered that. So it's a mixture of people thinking this is going to sell better and people just not thinking at all because the teams that are building this technology are predominantly young white and male and they have these blind spots where they don't even consider what biases they might perpetuate through the design of these systems. So you sometimes call a robot ethicist and you sometimes said we might need to establish a limited legal status for robots.

What do you mean by that? So it's a little bit of a provocation but my sense is that if we have evidence that behaving violently towards very lifelike objects not only tells us something about you as a person but can also change the world. And if you're a person but can also change people and desensitize them to that behavior in other contexts so if you're used to kicking a robot dog or you're more likely to kick a real dog then that might actually be an argument if that's the case to give robots certain legal protections the same way that we give animals protections but for different reason. We like to tell ourselves that we give animals protection from abuse because they actually experience pain and suffering.

I actually don't think that's the only reason we do it but for robots the idea would be not that they experience anything but rather that it's desensitizing to us and it has a negative effect on our behavior to be abusive towards the robots. So here's the thing that sort of it's worth pondering for a moment. If you hear for example that someone owns a bunch of chickens in their farm right so their farm their chickens they own the chickens and they're really mistreating the chickens swatching them harming them. You could sort of make a property rights argument and say they can do whatever they want with their property but I think many of us would say even though the chicken belongs to you there are certain things you cannot do with the chicken.

And I'm not sure it's just about our concern that if you mistreat the chicken that means you will turn into the kind of person who might mistreat other people. There's sort of a level there's a certain moral level at which I think the idea of abusing animals is offensive to us. And I'm wondering the same thing as true with machines as well which is it's not just the case that it might be that people who harm machines are also willing to harm humans but just the act of harming things that look and feel and sound sentient is morally offensive in some way. Yeah so I think that's absolutely how we've approached most animal protections because it's also it's very clear that we care more about certain animals than others and not based on any biological criteria.

So I think that we just find it morally offensive for example to torture cats or you know in the United States we don't like the idea of eating horses but in Europe they're like what's the difference between a horse and a cow they're both delicious. So that's definitely how we tend to operate and how we tend to pass these laws and I don't see why that couldn't also apply to machines once they get to a more advanced level where we really do perceive them as a life like and it is really offensive to us to see them be abused. And the level's advocate side of that argument of course is that people then say pressing a switch and turning off a machine that's unethical because you're essentially killing the robot. But we don't protect animals from being killed.

We just protect them from being treated unnecessarily cruelly. So I actually think animal abuse laws are a pretty good parallel here. You mentioned Westworld some moments ago and I want to play a clip from Westworld for those of you who haven't seen Westworld. Humans interact with robots in a robot that are extremely lifelike so lifelike that sometimes is difficult to tell whether you're talking to a robot or you're talking to a human.

In the scene that I'm about to play you a man named William interacts with a woman who may or may not be a robot. You want to ask? So ask? Are you real?

Well if you can't tell. Does it matter? So as I watched the scene and as I read your work I actually had a thought and I want to sort of run the start experiment by you which is that one of the spectrum we have these machines that are increasingly becoming lifelike, human like, they respond in very intelligent ways they seem as if they're alive. And on the other hand we're learning all kinds of things about human beings that show us that even the most complex aspects of our minds are governed by a set of rules and laws and laws.

And in some ways our minds function a little bit like machines. And I'm wondering is there really a huge distinction? Is it possible, is the real question not so much can machines become more human like, but is it actually possible that humans are actually just highly evolved machines? I have no doubt that we are highly evolved machines.

I don't think we understand how we work yet and I don't think we're going to get to that understanding anytime soon. But yeah I think I do think that we follow a set of rules and that we're essentially programmed. So I don't distinguish between souls and other entities without souls and so it's much easier for me to say yeah it's probably all the same. But I can see that other people would find that distinction difficult.

Do you ever talk about this? Do you ever run this by other people and sort of say do you tell your husband for example? I like you very much but I think you're a really intelligent machine that I love dearly. I haven't explicitly said that to him.

But when you go home from this trip. Yeah we'll see how that goes. Kate Darling is a research specialist at the MIT Media Lab. Our conversation today was taken for a live audience at the Hotel Jerome in Aspen, Colorado as part of the Aspen Ideas Festival.

Kate, thank you for joining me to be on Hidden Brand. Thank you so much. This week's show was produced by Rina Cohen, Tara Boyle, Renee Clark and Bartia, our team until Janish Manta and Maddy Piment. NPR's Vice President for Programming is on your run.

You can find photos and a video of Samantha, our Clio Dinosaur on our Instagram page. We're also on Facebook and Twitter. If you enjoyed this week's show please share the episode with friends on social media. I'm Tran Krapitantu.

See you next week. Starting a business is tough. Shopify makes it easier. Build a beautiful store.

Right with AI and manage marketing, inventory and shipping all in one place. Join millions who trust Shopify to launch and grow their business effortlessly. If you're ready to sell, you're ready for Shopify. Turn your big business idea into sign up for your one Euro per month trial at Shopify.ie.

Go to Shopify.ie. It's time to see what you can accomplish with Shopify by your side. At Vodafone you get more. Like more than more.

Get a brand new Samsung Galaxy S26 Ultra for zero Euro. Plus get more peace of mind. With an extra 12 month warranty when you switch to our unlimited bill pay mobile plan. Now that's a whole lot more.

Vodafone. Together we can. Subject to availability eligibility and switching to our 24 month 65 Euro per month on limited bill pay plan by 30th June, 26th. Each April the price of your plan will increase by three or fifty per month for full term C Vodafone.ie for slash terms.

Frequently asked questions

How long is this episode of Hidden Brain?

This episode is 48 minutes long.

When was this Hidden Brain episode published?

This episode was published on January 12, 2018.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Hidden Brain episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.