MSP125 [] Implicit Bias: How Machines Learn To Hate.

This is a download from the FM 89.9, the Business Station. It's Friday, my name is Jafsan Du, with protests sparked by the killing of George Floyd reverberating across the world. MSV takes a look at whether technology is part of the problem or the solution. Matt, welcome.

I guess we're throwing that technology is neutral concept out of the window today then. Hey, Jaf, how are you doing? Yeah, I guess we are. Honestly, today's topic is quite a difficult one for me to do.

For starters, obviously, I'm white, and although this show is called Matt's Blame, the last thing I want to do is spend the next half hour or so white-slaining racism to people. Now, we have covered technology and race and gender in the past, and I do try and stay away from things that are too directly political. I have a lot of my own views on things like climate change and whatever. I'm very outspoken about my beliefs on my personal pages, but this is a radio station.

It's not a place for me to push an agenda or to tri-nalianate people. But this is mostly a show about the ways that the business of technology affects culture and society. So I do sometimes have to talk in those kind of universal terms, rather than narrowing things down to specific groups. But today, we are acknowledging that technology isn't universal.

Well, yeah, a couple of weeks back, we were talking about technology's response to COVID-19, and we discussed accessibility, that a rapid transformation away from bricks and mortar stores to online retail, for example, imposes a much higher cost on vulnerable sections of society, the poor, the unbanked, the elderly. While things like the adoption of apps, it's not easy for everyone. I experienced that myself to a smaller extent a couple of weeks ago. I thought I paid my phone bill in full, but I'd switched to a more expensive data plan that included an Apple Watch, and that meant my credit limit on the phone account had reached 80%.

So the line was barred. Is this going somewhere? Well, yeah, because I was out at a mall and suddenly I had no data, and I couldn't use any of my apps. I couldn't even use the service providers app, because the service provider isn't sending out bills at the moment.

They ask you to look at the app. I couldn't even figure out why I didn't have data and couldn't use any of the apps that you have to use to get into the store. I had trouble getting in from the car park. I had trouble going into each store.

I was causing additional delays because I had to slow the line down because I was filling in an entry book or using a staff member's phone instead of just scanning a QR code and using my own phone. So not only are you embarrassed that you're holding up the line, you actually become a physical impediment. So that day, I only did the essential things, which is supermarket shopping and of course getting a nice blended and heading home to figure out what was up with my phone. But for some people, that is their permanent reality.

Maybe they don't have a smartphone. Maybe they don't have credit on the line. Maybe they simply have trouble downloading and using apps. There are a lot of reasons why technology doesn't work for everyone.

And there's also racial bias in those systems too? Well, in the example of me holding up a line, no, it's not in the technology itself. I imagine there were a couple of people in the queue who might have added my skin colour to my apparent stupidity. It's possible.

I mean, I didn't get that feeling. But there are racial disparities in the way that a lot of technology operates. So today, I wanted to look at some of those issues and highlight to some of our listeners who perhaps do still think that technology is neutral. And also because reading the tech press for the last couple of weeks has been a little strange.

I follow quite a few tech writers on Twitter. A lot of them are very politically and socially active and vocal. But a lot of the tech sites seem to look like business as usual. You might get one or two pieces relating to technology and the current global protests.

But then it sits next to news about Facebook co-founders, the Winklevoss twins, making a movie about themselves. It seems it's kind of a strange parallel. And you think they could be doing more? Well, look, I understand if the majority of visitors to your site want to read about the latest Nintendo Switch-like phone gaming controller, it might be hard to get them to click on an article about systemic bias in machine learning.

And to their credit, most of the tech sites are covering stories like Reddit trying to clean up its forums again, or senior Facebook staff publicly criticizing Mark Zuckerberg's stance on Donald Trump's posts at a time when Twitter has decided to monitor and label that kind of content more aggressively. But from what I've seen, it's not because the writers lack the knowledge of the sensitivity to write about these topics that I'm currently struggling to talk about. Is this something that the tech press and shows like this should be interesting? Absolutely, yes.

I mean, that goes back to my point about apps and accessibility before. You know, we're using technology to rapidly reshape society. Think of all the things that have become normal for us over the past 20 years, you know, that the internet, smartphones, forums, social media, think of all the changes to our social behavior that have come as a result of those new adoptions. Like dating apps?

Well, you know, that's actually a really good place to start because, you know, the dating app revolution happened long after I became involved in a long-term relationship that has resulted in marriage. So I have no experience of actually using data apps at all. So I don't understand the dynamics of chatting with strangers over text, exchanging potentially intimate photos for people you've never met and trying to figure out that whole hookups or relationship thing. So in theory, you know, dating apps should be a place where anyone can find someone.

And that isn't the case. Not always. You know, a 2018 study at Cornell University found that many of the top dating apps asked users to enter their own race and many allowed users to pick the preferred race of potential partner matches. Others allowed users to filter potential matches by race.

So we come back to that accessibility thing I mentioned earlier. These apps are supposed to make it easier for you to find an ideal match. Yet you can use exclusionary filters that exclude entire races from that search for the perfect person. A 2014 OKCupid survey found that black women and Asian men were generally rated lower by other users than other ethnic groups.

White men and Asian women tended to score most highly. Other studies have found that black users are 10 times more likely to contact white users on dating apps than white users are to contact other black users. And that's without the comments. Yeah, you know, some sites have started to come down harder on people's profiles explicitly stating know this race or know that race.

But yes, you know, racial abuse on the apps and the sites is well documented. A grinder, one of the leading dating apps had a reputation for racially insensitive comments and launched an education campaign called KINDER. I think that was back in 2016 or 17. An old school friend of mine, this is anecdotal, who now lives in the US posted comments on her Facebook page a few weeks ago.

And she was receiving a torrent of abuse from someone she'd been chatting to on a dating app and who obviously trawled her social pages and found evidence of things he disapproved of. And the thing he disapproved of was that she, a white woman, had dated and had relationships with people of color in the past. And, you know, there was just a torrent of abuse from this guy. But that's nothing to the kind of abuse that those people of color experience.

Even down to, you know, the supposedly well meaning, but very tone deaf, oh, you look so exotic to full on abuse from people. And what are the apps doing to combat this? Well, some of them take it very seriously, but not all of them are doing anything at all. One app that was contacted by a wide magazine, I'm not going to name them because they may have changed the policy since they routinely matched people with other users of the same race, even though no racial preferences were made during the registration process for that app.

Their justification was that empirical data showed that most people prefer to date within their own race. So the data sets are flawed? Well, it ignores a lot of factors. Firstly, you know, most countries were relatively racially homogenous until the last half of the 20th century when people really started to move across borders and across continents.

And in many cases where there were large numbers of people of other races living alongside one another, countries like the US, laws were often enacted to prevent intermarriage. Or local communities would take the law into their own hands and run off or even kill, generally speaking, the member of the couple who came from the ethnic minority. So historical data would likely confirm that people date within their race because those were the only choices that people had. But if you allow users to filter by their preferred race, it's going to skew those data sets as you move forward as well.

Machines aren't smart. You know, we love to give them that terminology, but they're not thinking creatures yet. I don't want to go into the AI part too much at the moment, but we will come back to AI and bias later in the show after the break. So there are steps that dating sites and apps can take to eradicate some of these biases.

Well, you know, we have this very weird relationship with data. We view it as this kind of absolute power. And a lot of people, I think, fail to realize that data is very flexible. In the sense of what is and isn't true?

No, in the sense of how we actually use the data. So recently, I was sent an animated data visual of COVID cases against other illnesses, the kind where you see the number of cases accumulate. So some of my friends pointed out that the diseases it was compared to were it was a highly selective choice. It ignored conditions like strokes or heart disease that result in large numbers of deaths over sustained periods.

But the real point of the chart was not to say that this illness was the worst illness that has ever before in humanity. It was to demonstrate the exponential spread from a handful of deaths to hundreds of thousands of deaths over a period of just a few weeks. In other words, the dating services could serve up the R data in different ways. Absolutely, because there's no need to tell the machine to use race or physical characteristics to serve up potential matches.

Those data sets can be broken down into thousands or even millions of individual units. And that can create new intersection points. That's one of the reasons that I get so excited talking about data. It's not static like a lot of people think.

And when I talk about it like this, and it's not about manipulating the data, it's about approaching it with imagination. It's about using the information that has been reinforcing a problem to actually help to solve it. When we come back more about the machines that discriminate, you're listening to MSB, the event 89.9. Benchmark for managers, BFM 89.9.

Welcome back. It is Friday. My name is Jasana together with the College of Ops Mat Amortech. Before the break, we started to talk about machine learning and biases that can be baked into artificial intelligence.

Yeah, but before we go there, let me give you one more example that's related to accessibility and bias. So I like using speech recognition software, especially when I'm doing notes for shows and talks. I find that it comes out more naturally, it's better reflects the way I speak. Your recent attempts have essentially been gibberish.

Yeah, I mean, I sent you one script that you basically sent back to me and said, what on earth is this? But that's kind of the point I'm about to make. I used to use apples baked in dictation, and it was pretty good for British English. Now, since Apple introduced the Catalina OS, it has changed the way that dictation is processed.

And now it seems to support British accents a lot less well. I'm not sure why, but there are plenty of other people complaining about it online too. So to get it even halfway accurate, I have to switch to US English and start dictating with a fake US accent. The last one you sent me wasn't even halfway accurate.

Exactly, which tells you how good my US accent is. But I was absolutely outraged that I should be treated like that by the company. But that's also a sign of my privilege as a British English speaker. That kind of treatment is routine if you don't speak US or British accented English.

You're excluded from a lot of the advances in natural language processing, and yes, I do know that English and its dialects are not the only ones supported. But they do attract a lot of the attention and the funding, especially as natural language processing is becoming increasingly important from the Alexa type devices we have at home to public information kiosks and terminals in an era where touch screens may increasingly be seen as a point of infection as an infection risk. So it comes back to that point of technology working for everyone and not just for everyone who has an accent like mine. Looking beyond your white privilege?

Well, you know, let's go back to a story I think we covered a couple of years ago. I can't remember if it was here or on geek. So it was about an AI that was in use in the US prison system. The idea was that the system would make recommendations on whether someone was likely to reoffend based on data.

So it was thought that removing the human element where there might be institutional biases at play might or should lead to a fairer system. A pro-publicer report in 2016 reported that the system was weighted against black defendants. It assumed them to be at a higher risk of recidivism than the data actually showed. Similarly, it weighted white defendants at a lower risk than the data actually showed.

Similarly, predictive policing algorithms have been found to unfairly target low income, racially diverse neighborhoods, by which I mean, of course, non-white irrespective of the actual crime rates in those neighborhoods. In other words, it's really about the quality of the data that's going in. To an extent, but there's also a lack of minority representation in the tech industry in general. So it isn't always just about the data.

Sometimes it's about conscious and unconscious biases of the people who are coding and putting the information into those systems. So this is another story we mentioned a few years ago. A website called Beauty AI held an online beauty competition with a twist. The twist was that it would be judged by an artificial intelligence.

Now it received more than 6,000 entrants from over 100 countries. And the idea was that the system would judge those that most closely resembled the ideal depictions of human beauty. So it would use criteria like facial symmetry. Now of the 44 winners selected, most were white.

There were a few people who were Asian, but only one of the winners had dark skin. And because the AI underpinning the competition was supported by Microsoft, it attracted considerable outcry at the time. It was both a data and people issue? Data definitely.

There were relatively few images of non-Europeans in the data set that the AI had been trained on. So the machine learned its concepts of beauty from white faces. A wider data set could have led to very different results. And that's where the humans come in.

The machines have to be fed or pointed to the data. If they aren't pointed towards truly representative data samples, then those results will be skewed and biased. And that then makes expanding the data set in the future more problematic. If the algorithm has already decided that that data was the foundation, if it isn't actually programmed to learn from dynamic data that encounters, it will simply filter all results based on those incorrect starting assumptions.

I've talked a lot about dating and beauty, but the real world consequences of this kind of data are a lot more widespread and a lot more insidious. For example, an algorithm might block credit or loan applications based on your address or some personal history details that are rooted in race rather than your actual risk assessment. I mentioned the policing systems already, but we have to make sure that we don't overlook any of these issues with the information we put into society. I think we've mentioned bias in facial recognition systems a few times as well.

Yeah, and we have an updated story on that subject in Geeks, but a 2018 survey by MIT noted that many of the latest gender recognition AI systems were as much as 99% accurate as long as you happen to be a white male. They were generally less effective for women as a whole, and for dark skinned women, that accuracy dropped to 35%. So you can see that disparity for a white male like me, 99% accurate for a black female, 35% accurate. And what would some of the real world implications of those results be?

Well, facial recognition is only as useful as it is accurate. In law enforcement situations, we see inaccurate recognition leading to false positive readings. Tests in the UK, test by UK police on facial recognition systems found a successful match rate on only 20% of their scans. In one instance, somebody was identified as a suspect and stopped and searched before it was discovered that it was a false positive.

But with systems like this increasingly becoming the norm around the globe, we've risked the machinery of society treating entire social and racial groups as potentially criminal or as high risk, simply because we've implemented the use of machines or data that classifies people like, basically, me as being safe or ideal or preferred. And that's the problem because the opposites of those words are dangerous, wrong and undesirable, and the machine will be applying those to everyone who doesn't look like me. Some of the studies and examples you mentioned are in 2015, 2018, 2019. Those are practically aeons in terms of machine intelligence training and development.

Yeah, well, you know, there have been enormous strides in many situations. Some have been good, some arguably less so. I've mentioned this example a few times systems in China that have been used for COVID-19 enforcement. The systems have become extremely accurate at identifying people wearing surgical masks.

But there's a fear that those same technologies could be deployed against democracy protesters in Hong Kong who have gone to great lengths, wearing things like masks to avoid being identified. And this is where we cycle back to Microsoft's artificial intelligence tools. The story about the pop group Little Mix? Yeah, so this came out this week.

Microsoft's MSN portal. Do we still have portals? Microsoft's MSN posted an image of the wrong member of the band. Now it mixed up to mixed race members of the band.

The company was called out and criticized by the band and by other people on social media. And it later came out that this wasn't human error. The photos were actually selected by machine intelligence. So it was reported maybe a week or so ago that Microsoft is planning to sack many of MSN's human editors and replace them with software.

Now we've seen a few other tech and social media companies trying to do that in the past with predictably disastrous results. But this shows the problems inherent in automated content systems. They will reflect the biases or the absences of information that have been programmed into them. And this leads to some bizarre results.

In what sense? Well, MSN publishes and re edits content largely from third parties under content sharing agreements. So this story was broken by the Guardian, which is one of the sources that MSN pulls content from. So the Guardian reported that MSN was trying to limit reputational damage to its AI systems by not publishing the Guardian report and presumably reports by other content partners.

According to the Guardian report, that meant that the human editors were instructed to override the content choices of the AI that was destined to replace them and remove this story. But they were also told that the AI might try to override the human editors and their attempts to spike the story because the AI determines that story to be of interest to MSN's readers. So it becomes this Kafkaesque situation where you're just going around in circles. A machine publishes something, it makes a mistake.

People report that mistake. The machine then tries to publish that story against the wishes of the company that it is publishing stories for. Human beings try and spike the story. The machine overrides and overrules the machine and the machine can then try and override the people again.

We're just going around in circles and sentences. And if none of that makes any sense at all, that's because to me it sums up our attempts to address racial bias within text systems. If your starting point is incorrect, then the failures just get amplified the further through the system you continue and they amplify and resound. So be honest, I lost you somewhere in between.

But where do we go from here? Well, you know, this is by no means an exhaustive account of racism and tech. This is all that I could fit into this sort. As I mentioned before, there's the lack of representation of many races which is seen throughout the tech industry.

You know, we haven't gone into things like the role that technology plays in fueling and spreading hate speech. The tone-defriactions of tech companies and lawmakers when it comes to regulating and controlling that kind of hate speech and activity. I mean, there are just so many examples that I could give. But I know that not everyone wants to be an activist.

Not everyone wants to be part of the solution. But at the very least, you know, get out of the way. Don't be a part of the problem. Don't tell that crazy uncle that he's entitled to his opinion, telling him he's offensive and, you know, unfriending.

And I'm a middle-aged white guy. Don't ask me what to do about racism. Ask somebody who experiences it. But then don't dismiss that person when the conversation becomes uncomfortable for you and your point of view.

MSP, you know, going through, I guess, you know, difficult conversations. But, you know, at the end of the day, I guess, you know, it's important to also understand how machines are, you know, have learned to discriminate. And yeah, interesting points. Do stick around though, because geeks have a story that is, you know, following up on what Matt was talking about, you know, the facial recognition stick around.

If you missed any parts of this episode, you can download the podcast available on the BFM website or the BFM app. Geeksquox, after this, BFM 89.9. Thank you for listening to this podcast. To find more great interviews, go to bfm.my or find us on iTunes, BFM 89.9, the Business Station.