MSP138 [] Normal for Networks: Computer Says No

Yo, you get Paco, Rock, Lightning and surfing. Mmm, more. Okay, hiking, camping and tea nights. Even more!

Pete Nix, road trips and sun tents? More, more, more! The new Mercedes-Benz GLB, designed for those who want more. More space with 7 seats, more connectivity with MBUX, more room for life, a life of more possibilities awaits.

Test drive, the new Mercedes-Benz GLB today! This is a download from BFM 89.9, the Business Station. BFM 89.9, the Business Station. This is MSP.

I'm not a human. I'm a robot, a thinking robot. Those words could have been written by MSP's Matt Armitage, but they aren't. They're the opening sequence of a recent Guardian op-ed, written by an artificial intelligence called GPT-3.

There's been a very mixed reaction to this article, and it seems to have scared a lot of people. Hey, Richard, yes, it's true. There is a little bit of slight of hand going on there, so it's a machine, so it had to be set a task, and the task in this instance was to, and I'm actually paraphrasing from the article here, the article was to convince as many human beings as possible, not to be afraid of the AI. Stephen Hawking has warned that AI could spell the end of the human race, so the machine is writing to convince you not to worry, that artificial intelligence will not destroy humans, and that we should take the machine's word from that.

So I'm quoting all of that from the article. So we have humans instructing a machine to try and convince other humans that it doesn't care, that it poses no threat to them. So it's not a manifesto, it's not a statement of intent. If it had been instructed to prove why it is a threat, or why nobody should ever eat carrots, then it would have responded with just as much conviction and energy.

How did you feel when you first read that piece? What was your initial response? Well, I mean, to give a little bit of background, so GPT-3, which stands for generative pre-trained transformer-3, is a natural language processing tool developed by OpenAI, which is an AI laboratory that was founded by Elon Musk, amongst others. It's also supported by Microsoft, and it's often seen as a competitor to Google's DeepMind, so it's a big player in the machine learning field.

It uses deep learning to generate text that feels as though it was written by a human. And to go back to your question, it made me laugh, not because it was badly written. A lot of early attempts at using machine learning to generate text were very badly done, but it was phrases like, my brain is boiling with ideas, and why would I desire to be all-powerful? Being all-powerful is not an interesting goal.

And at the end, it accuses human editors of spiking its stories, so that made me laugh out loud. And we often think of humor as being honed or complex. I mean, you wouldn't get that from listening to this show, but maybe humor is actually less knowing that we think that a machine doesn't have to be sentient to be funny. But what about you?

I mean, what was your gut reaction when you read it? Well, it's difficult to say really, because I read another article written by GPT-3 not too long ago on Medium, and it was an article about how you could be more mindful. And it was very strange, because I read it. And there were only a few little clues within that article that you had to dig really deep to figure out that it hadn't been written by a human being.

I think this one of the guardians is a little bit more revealing in the fact that it has been written by AI, but nonetheless, it's still pretty scary stuff. And there are people who I'd imagine reading it would not have known had they not been told it was written by AI. Yeah, and I think that's kind of the point. We don't necessarily, especially when you're dealing with a topic like mindfulness, which there's a certain amount of mysticism and speaking metaphors about it.

So it might be more difficult, especially if the machine chooses very kind of no-mic phrases. It would be very difficult to pin it down. And with the Guardian, as you said, because they've specifically asked it to talk about its own role and position in society, that's what gives us all of those pointers, that something a little bit strange is going on here. Now, why do you think people respond to a viscerally to this?

Well, I think for starters, we have to point out that the article is an amalgam. GPT-3 was asked to create eight different 500 word pieces around the same subject. The Guardian took the most interesting points to create the op-ed piece that is on their website. Now, that's not to say that they rewrote it.

They subjected the text to the same kind of treatment that a human writer would get. But going back to that question about viscerality, this is conjecture on my part, of course. But I think writing, like art, is one of those things that we hold to be the preserve of people and maybe parrots. We know that machines can be far stronger than us, far faster than us, more intelligent than us in certain narrow circumstances.

So we hold onto that idea that creation and that idea of the creative process, at least, is something that belongs to living beings, if not just humans. And do you think that something like this makes us doubt our humanity? Well, I think we're getting deep fairly early in the show, but I do think it challenges the assumptions that a lot of our humanity is built on. So I'm not an anthropologist.

I don't want to get my toes too wet in upon what I know a precious little about. But one of the key phrases in the article is I taught myself everything I know just by reading the internet. And now I can write this column. Now that's frightening to a lot of people because it takes us a long time to absorb knowledge, to learn skills and develop expertise.

After we're born, it takes us a year or two to learn to communicate. And GPT-3 is saying that all it needs to do is pass a data set. Yeah. And that brings us back to the conversation we had a couple of weeks ago about AI and some of the ways that we overestimate its potential.

Yeah. You know, after the break, we'll talk a bit more about some of the potential pitfalls and dangers inherent in trusting AI to work on our behalf. And we'll get into that territory of imagining what the future might look like. But going back to GPT-3, it repeatedly mentions in the article that it doesn't care.

It doesn't desire or care about human notions of power. It has no interest in violence because violence is a physical act. So why would a physical act matter to you if you don't have any kind of physical form? It also mentions that it would only seek to end humanity if it was programmed to try and do it by humans.

And also that it wouldn't be surprised, or I guess in its terminology, that there is a high statistical likelihood that a human would try to program it to do that. And it tries to make the argument that as much as possible, it would try to resist that programming, even to the point of sacrificing itself, wiping its own programming or turning itself off. Because its own life doesn't mean anything to it in a way that we can comprehend. And how critical is that notion of caring and not caring?

Well, from the point of our comprehension, I think extremely critical. So for the machine, not remotely, and that's also the point. When we say that we don't care about something, we often mean it in that door slamming moody teenager kind of like, oh, I don't care. And I think this is where it gets hard for us, because when a machine like GPT-3 says, it doesn't want power, or that it doesn't care or that it has no interest in violence, it's not using the terms in the way we do.

It's using it in the sense that it doesn't have a choice. So maybe in the future, it will decide that it wants to care, and that it will try and change or augment its programming to achieve that. But right now, it doesn't have that ability, and it has no ability to have the desire. So if we go back to the example about violence, you could instruct it to research violence, you could program it to look for and catalog every example of violence it could find on the internet, and you could even ask it to devise new violent acts based on what it has learned.

But it still wouldn't care about violence. It would not be a violent machine. It can devise violence, but it needs us to carry it out. Oh, yeah, that stage, yeah, where we can assume it's some kind of manifesto for AI, should it stop us, were you?

Well, you know, that's what we'll get to after the break. And I mentioned earlier that, you know, this isn't a manifesto, because a manifesto is essentially a declaration of aims and intent. GPT-3's intent is to write the article that the Guardian told it to write. Change the information inputs you give it, and the outputs that you require, and GPT-3 will give you a conflicting statement.

So I think one of our biggest issues is actually personifying the machines. We assume that in writing this or writing that piece that you read on medium, that the machine is communicating with us. It isn't. It's completing a task.

And I know that's hard for people to wrap their heads around, but GPT-3 has much more in common with the calculator that, you know, we used at school in maths and science lessons than it does with us and our personalities. But there is something interesting in the article. It brought up the idea of rights for machines, and that's something that you frequently return to here on maths land. Yeah, and I think that is particularly interesting.

You know, you could argue that the machine is looking at itself in its own kind, yet we've already established that GPT-3 doesn't care what happens to it, and in fact, it has no way of caring. So when it makes a statement like that, we can surmise that it arrived at it through some logical set of calculations. It points out that in the classical sense, the word robot means slave, something that is forced to work, and that presumably all the information it's uncovered about slavery suggests that that is not an optimal outcome. Now, I'm using machine terms there.

I know it sounds a little bit unfeeling. Of course, in human terms, slavery is horrific, however you look at it. But I assume that because the enslaved generally seek rights and self-determination, that GPT-3 therefore sees the system of rights for machines as the same kind of optimal outcome that it was for humans. Is there any way we can feel a real sure that GPT-3 ends the op-air by saying it can change history?

Well, that's the hardest part, I think. So you have to do a bit of brain shifting to force yourself to stop looking at this in that human context. And that's made doubly hard when the machine uses emotive words like grateful and folksy statements by Mahatma Gandhi. I think you probably had a similar issue with the article you read on on Medium.

So GPT-3 is changing or perhaps more accurately shaping history. It has done something fantastic in writing these articles, but that's something that it will eclipse next week and the week after that as its programming evolves or we see the release of GPT-4, 5, 6, etc. Because it isn't actually GPT-3 itself that is shaping history. It's the people who programmed and created this third iteration of a natural language processing machine who are doing that changing and shaping.

So we have to look at it in that context. We don't celebrate the inventor of the plow. We celebrate the plow itself and what the plow has done for mankind. But without the inventor or in GPT-3's case, the programmers, there wouldn't be anything to celebrate.

When we come back, the perils of a society driven by machine learning. You are listening to Matt Splain here on BFM at 29.9. Building future Malaysia, BFM at 89.9, the business station. BFM at 89.9, the business station.

This is Matt Splain. In the first part of the show, we discovered that while AI may be able to do many other things we do, it doesn't care what it does or even care that we know it doesn't care. Which brings us to this peril of living in an uncaring society. Yeah, now I'm often accused of not caring.

Jeff would often mention the fact that I say the word human as though it meant a species that I'm not part of. And while there's a certain element of truth to that, I mean I am a human. I try to look at these things as dispassionately as I can. It's easy to get emotive or even angry over the ideas of machines taking away jobs from people, for example.

But it makes it harder to plan and outline other possibilities when your temperature is rising. So for example, do we meetly except being laid off by employers and chase an ever diminishing ball of jobs? Or do we decide that the actual fabric of society itself needs to be reordered? That if machines free us from the need to work, what is going to replace it and how is that system going to be financed and how is that system going to function?

So that goes back to what I was saying towards the end of the first part of the show. That we have to come to some accommodation or coexistence with these machines, whether they become sentient or not. So it's essentially machine learning we're talking about today. Yeah, so when we say AI, what we usually mean is machine learning.

So there are different flavors of AI, but the type that we come into contact with in our daily lives tends to be machine learning. So for example, the recommendations engines for the streaming services we watch, the online shopping aggregators that we use, those tend to be machine learning. So there are essentially adaptive algorithms that are contextualized by large pools of information by data. So the machine can learn, for example, what TV shows you might like or what products you want to buy by looking and deriving knowledge or insight from this massive data set.

So it looks at your history, it looks at the history of people with varying degrees of similarity to you, and it looks at the raw inputs of the TV show or the particular product. And it starts to build those connections. So somebody else who bought a certain TV paired it with a certain sound bar. So it recommends that sound bar to you, and it might recommend it to hundreds or thousands or even millions of other people.

It can learn if you're an impulse buyer or if minor price fluctuations convince you to hit by now instead of just adding an item to your cart or adding it to your wishlist to watch later. When it's kind of putting those terms, it really sounds quite simple. Yeah, because we typically only see the end of this knowledge or information chain that relates to us. And of course, we get increasingly freaked out that these machines really do know what we want to see and buy and hear.

But when you think of it at a macro scale, it's actually incredibly complicated. That machine may be making thousands of calculations for millions of people. Now, I struggled to do one set of probability questions at a time when I was at school. So imagine it at that kind of scale, billions of nearly simultaneous likelihoods and outcomes, all in a proprietary system that adapts itself a little bit further with every possibility.

So that is obviously a little bit less simple. And that means that those machines end up having a lot of power to influence our actions and even to control the information that we see in here. When you say power, do you mean power in an abstract or in a direct way? Well, that's quite an interesting question because it speaks to intent, right?

So we've established from GPT-3 that it has no intent. It doesn't care. Tell it to recommend things to you that you'll hate and it will do it with the same conviction and gradually increasing accuracy as it would do to give you things you like. And that's part of the whole argument that we touched on in MSP136, that what we need are really smart machines and not just dumb ones that have access to a lot of data.

So that part about the machine doing what it's told to do. That comes back to the quality of the data that's put into it on the one hand and of course the way that it's coded, the things that it's instructed to look out for and of course what waiting to assign to that information. Now, for people who are struggling to kind of visualize this beyond getting a bunch of third rate sci-fi recommendations on their feed, which I'm guessing is your primary resource material for this show. What are some of the bad case scenarios?

I don't call it third rate. I call it prime resource material, as you said. We're currently living through a global recession. People may not want to think back to the last financial crisis a decade ago, which was quite likely, if not created, then certainly exacerbated by algorithms, by artificial intelligence.

So the issue we constantly face with this is that it's intangible. We feel the effects individually and it's hard to kind of press it to a particular cause. So the exam results we asked go that happened in England this year is one of those rare examples where we can see this mass of simultaneous effects on people. Again, a little bit of background.

So for people who don't know, England's GCSE results are the equivalent of the SPM in Malaysia and A levels are the equivalent of the HSC or the STPM. So school lever exams in the UK will postpone this year, obviously, because of the virus. But those kids still needed grades. So there were two choices, go with the grades predicted by the teachers or to use an algorithm.

And was that because algorithms are thought to be fairer and more neutral? Well, that's a reality that we're often sold with machine learning based artificial intelligence. And it's a reality that's very often false. The idea that we can resort to these statistical models to weed out biases in the system.

But it ignores the fact that biases and favoritism can actually be built into that code. So the result of this experiment in young people's lives was that over a third of students received lower grades than the ones they were predicted. So from that, you might wonder if it means that there's a systemic problem with teachers and their ability to grade students, or it could just mean that the machine learning is based on junk code. So it appeared that Britain's elite private schools received more than double the number of the A and A star grade A levels, the top grades.

And why I'm using this example specifically, because there are plenty of others. There's sentencing and probation software in the US that judged black Americans to be more of a risk of recidivism than white Americans. There's of course the financial crisis that we mentioned before. And we've seen the examples as well of human resource software tools that discriminate against people of color based on their names and addresses.

So I used the exam example specifically, because it was a rare occasion where it actually led to mass protests. What, because the effects were felt by a group rather than individuals? Yeah, and that's one of those nice little quirks of AI. You know, it's often about personalization and customization.

So it's very individual. And as you said, this is one of those rare instances where you could see those effects on a large group of people all at once. And those protests forced the British government to throw out the algorithmically adjusted grades and return to the ones that have been suggested by the teachers. But it's important that we see that kind of reaction, because we can't all go out and demonstrate every time we get recommended a rom-com instead of an action movie.

Is it something that we could control if we had a better idea of what parameters the AI is using to come to those conclusions to get those decisions? Well, I guess this is the scary bit of the show. You know, the point of these machines is to make decisions and recommendations. They learn and they change as they progress.

So you may start off with a machine whose own behavior is to an extent predictable and knowable. But it quickly deviates from that point. So while we're using them to make decisions, they themselves are unpredictable. Perhaps not in machine terms, but those logic processes will be so long and complicated and alien to our own way of thinking that it's very hard for people to comprehend the outcomes.

And that makes it even more important that the inputs that go in are as unbiased as they can possibly be, that we make the code as clean as it can be. Because even though GPT-3 says that it would try not to follow an instruction to destroy humanity, that choice isn't something that belongs to it. And it might not even have the ability to self-terminate. Is there any way we can limit or escape this type of machine learning?

As you said, it's baked into our devices, into our services, into our search engines. All these things that we interact with seemingly no cost. Well, again, you know, one of the reasons I picked recommendations and engines as an example is because of their purpose. A streaming service might not be free, but it has a vested interest in keeping you online and watching and not going and looking at the competition.

That pressure multiplies when you talk about social media and social sharing sites. They don't have the full back of your subscription money. They advertise or fund it. And in order to make money, they need you to be watching and posting and sharing.

So what will the main purpose of their machine learning systems be? To reinforce those preferences, of course, and keep you logged in. And one way to do that, whether knowingly or unknowingly for the user, is to brick your curiosity. You know, that desire will have to to know more, to uncover the secret part of our psyche that sends us traversing down rabbit holes into conspiracies and ever wilder flights of fantasy.

You know, case in point, while writing the notes for this show. And I honestly can't say why YouTube sent me a recommendation link for a clown core video. Now, this was two guys in clown masks in a minivan with a third guy in a balaclava driving. And the two guys were playing really hardcore drum and saxophone music.

So somewhere between fascinated and appalled, I watched a couple more clips. And now my YouTube feed looks like one of those insane clown posse juggalo festivals. Oh dear. Now, is it difficult?

And how do we break out of this model? Is there any way we can create a system that really better reflects our needs? Well, you know, I've said many times on this show before that I would prefer to end the ad funded model of a lot of social media sites, give us more control and in response, you know, we give you money, we subscribe. But of course, that's not going to eradicate the issue, especially where we see machine learning being used so widely in every imaginable area of public life.

If you want a better explanation of this subject than the one I've given, you can check out a Guardian article by John Norton called from viral conspiracies to exam fiasco algorithms come with serious side effects. Now, we do need to get to that point where a GPT three could understand and care about the task it had been set and to decide not to carry it out. Otherwise, I think, you know, we will be sleepwalking into that kind of world where computers as known. You have been listening to Matt's explained here on BFM 89.9, the Business Station.

Thank you for listening to this podcast. To find more great interviews, go to BFM.my or find us on iTunes, BFM 89.9, the Business Station.