This is a download from the FM 89.9, the Business Station. It's fun Friday, my name is Jeff Sandhu. About three months ago, College of Ops met Amatage, claimed that we will soon be living in the age of Amazon, a world ruled by mega companies who control every facet of global trade in our economic activity. Has he had a change of heart?
Has the algorithm running through his thought process short circuited? Well, to find out, we'll have to let him met Slane. Now, we're not actually in the studio together today, aren't we? No, your new studios have this really amazing facial recognition software, and they won't let me in.
This is a problem I've had for years, but for some reason, facial recognition software identifies me as a basket of kittens. So I'm recording my side of the show from Kuchapop's bunker inside a dormant volcano, so apologies if the sound is a bit weird this week. So you've changed your mind about Amazon and the mega corporations. Would you be annoyed if I started yet another episode by saying yes or no?
I think it's too early to say that I'm wrong about Amazon, just as it's too early to say that I'm right. What I'd like to do today is introduce some new variables into the argument and listeners control their own conclusions from there. Now, you mentioned algorithms in the introduction, and that's really what I'd like to talk about today. It's the intersection of our digital world, these massive IT powered corporate colossi and the uncertain growing power of code.
This is something that you've come back to quite often this year, the idea that we're under threat from machine intelligence. Yes, and thank you for not calling it artificial intelligence. It's probably quite important to make that distinction here. We often use terms like artificial intelligence, machine intelligence, machine learning, deep learning and others quite interchangeably.
Now, I'm not going to go into that side of the argument too much today. If you are a bit confused about the terminology, there's a piece on medium.com by a guy called Callum McClelland, called The Difference Between Artificial Intelligence, Machine Learning and Deep Learning. It's a quick read and I think it's a great primer and we'll post the link with the podcast. Why is it important that I didn't call it artificial intelligence?
Because we've had various people calling artificial intelligence a threat over the last couple of years, including one of our old friends, the man who likes electric cars and space rockets and makes weird comments about cave rescuers. Now, if you've been following the shows this year, you'll know that it's not artificial intelligence in itself that concerns me. It's the fact that the AI or machine intelligence or the deep learning, whatever you want to call it, it's the fact that the systems we have now are not intelligent enough that actually worries me. The systems that we are releasing into the world that have this incredible ability to affect our lives, but have little or no ability to reason or to question.
A bit like a robot army in one of those all 1950s B movies. Yes, I've mentioned on the show already, you know, I've been rereading a lot of Philip K. Dix short fiction this year, which is absolutely breathtaking in its scope. You can see that a lot of it is him indulging in thought experiments and he's looking at the dangers posed by mindless automatons and that kind of comes up fairly frequently.
I'm sure that he would be absolutely fascinated to see the world that we live in today. And I'm wondering what he would think of the algorithms that are increasingly operating in the background of our lives. We've covered quite a lot of this ground this year. What are you bringing that's new to the table here, Matt?
Just some additional insight, I guess. I mean, I love the way that these subjects are always evolving. As a son, knowledge of them, I'm not a machine learning expert, but I do enjoy adding this knowledge to the evolving equation that is the world we live in. That's the thing about trying to imagine what the world of tomorrow looks like.
It's like this reverse butterfly effect. Every time you think you've got a handle on it, you learn something that distorts the picture. Or there's another technological or social revolution that up ends your mental image. It's constantly fascinating.
Yeah, if you're the one watching it rather than being at the receiving end. That's a fair point. It's like when people ask me if I'm worried about machines replacing all our jobs, and I say no. It's not because I think I've got the resources to survive, or I think I've got some amazing latent talent for exotic dancing that's going to make me immune.
I know that my way of life is at risk as much as anyone else is, but I think that my curiosity kind of overwhelms that fear. So then why bother to change things? Why not sit back and just let it happen? Because that's not the nature of the way we live.
Human beings aren't very good at being passive. I'm rereading Terry Pratchett's third at the moment, which is about two races of people who periodically get together to re-fight ancient battles and do it in a thoroughly deadly way. We can only be passive for so long. We like to tinker and change things.
Sometimes we do a good job and sometimes we don't. And machine and artificial intelligences are part of that evolving process. Can we change things? Can we influence the way the world is turning?
I guess that's a big part of the reason for today's show. You know, we talk about feeling impotent or powerless in the face of big corporations. And I've used this show many, many times to say that if we take a more active role, we can influence those companies. We can make Facebook and Twitter listen to us simply by either threatening to not use or actually not using their services.
And you're beginning to doubt that? No, I still think that we can make the companies move. It's not clear how much power we would have to do that in a world controlled by a giant mega company like Amazon. But for now we still have that power, that sense of agency.
So what's changed? I think it's the growing realization that the companies themselves are not really aware of what they're doing when it comes to algorithms and machine intelligence. That they're potentially unleashing forces that can only be controlled if we decide to switch off all the systems that connect us. We had our first inkling of the power of the systems to disrupt our world during the financial crash that happened in the noughties and the early teens of this century as the finance industry started to roll out high frequency trade software.
Ten years after that crash, and economists and analysts are still arguing about the causes and the role that technology played in that economic upheaval. You think that the confusion is an indication of the power that code has? Yes, because it's the first stage of a revolution or a technological evolution where we're designing systems that we don't fully understand. If you think about the example of a lot of tech-fueled disasters, maybe an air crash or a failure at a nuclear power plant, there are systems in place to prevent those disasters.
Each circuit or system has its own sensors and detectors, it's failsafes and shutups. Yet we still have occasional meltouts. Well, no system is foolproof, but at least the systems are designed by us. If something goes wrong, we may not be able to prevent that disaster and we can usually figure out what went wrong to try and prevent it from happening in the future.
And you think that we have passed the point with machine intelligence we have today? Very much so, I think, like with the last financial crash, when something goes wrong, we don't really have a way to understand what the algorithms did and what actually went wrong. We like to think of software programs as being very straightforward, very black and white. Ultimately, for a lot of us, it helps to have that image of switch on or switch off.
Yeah, we like things to be boiled down to a few points. Is there anything wrong with that? No, but we have to accept that the overview isn't the picture. It's actually the thumbnail.
It's the link that gets you started. The real image has far more depth and clarity. So if we go back to that article at www.medium.com, we like to have things boiled down to a few points. That article is maybe a four or five minute read.
And it gives you the basics. Certainly, it does so a lot better than I could. But at the same time, this is an area so complex that thousands of the world's finest minds spend their entire working lives exploring it. And even they can't tell you with any degree of certainty what happened during that financial crash.
So where does that leave us? We'll get into that more after the break. But before that, think back to all the breadcrumbs like Mark Zuckerberg announcing that he had no idea that Facebook's algorithms could be used to hoodwink a country or sway an election. We know that those algorithms have behavioural characteristics.
And you believe it? On the whole, I do. Yes. We think of software and programs and algorithms as being linear and limited.
But a lot of the systems we use today are designed to adapt to changing conditions. They are dynamic. They shift with our taste. They analyse news and cultural events.
But they don't interpret them in the same way that we do. So we're kind of designing a machine zeitgeist and not a human one. Surely the designers know the parameters they're programming? Yes, right now we are still in control.
We can still look at the code even though it is a mammoth task to review it. Facebook has been able to examine its code and make changes that they claim will protect us from further attempts to influence elections or sway public opinion. Where we have even greater problems is when these competing bits of code interact such as within the financial system. Those codes are designed to create profits for their authors and not to create stability.
So we're on the cusp of this next step already where far more powerful and freely operating algorithms are about to be unleashed. So if you go back to the nuclear power plant analogy, when the meltdown happens, there's no way for us to figure out what went wrong. The machines are operating with a language and a code of conduct that we have no way to comprehend. Well, after the break we enter the world of webinies code.
You're listening to Fun Friday with me, Jasant Dutka with CollegeBotsnet. I'm going to be right back BFM 89.9. Busy finding money BFM 89.9. And we're back and this is Fun Friday.
My name is Jasant Dutka with CollegeBotsnet Amat H man. Before the break we were talking about some of the examples of algorithms behaving badly. I think we touched on the financial crisis and the high frequency trade algorithms. You talked about it with Richard a few shows back.
That's right. Before the break I was saying that companies like Google and Facebook are still in control of their algorithms. So those high frequency trade algorithms, which are still a major part of the financial and trading sector, are an early indicator of where the technology is headed. These are pieces of code that are specifically designed to thrive on volatility.
They are designed to go head to head with other algorithms and try to outwit them. So their role is to try and distort reality and exploit any confusion that they create. And that's how they're physically designed. And because they're adaptive, we don't know where they're headed.
And neither do their authors. We suddenly find that there's a gap between the code that we create and control and the code we need to control new and complex systems. OK, that might need a little bit more explaining here. We're in a really very strange place right now.
So you take the fatal autonomous car accident earlier this year, which we've talked about on the show many, many times. So the lady who was hit was probably killed because the algorithm couldn't correctly identify her and predict her behavioral characteristics. So ideally the human driver should have taken control and prevented the accident because the software running the car simply wasn't clever enough. Which brings us to where?
There's a really cool article on the Guardian written by the technology journalist and author Andrew Smith called Frank and Algorithms. We'll post the link here. It's definitely one of the most interesting reads I've come across this year. Well worth half an hour of your time.
According to the article, we are about to enter a demimonde of powerful and unpredictable code, code that is capable of determining its own rules and laws, but is done by any measure of human intelligence. Neil Johnson, a physicist at the University of Miami who specializes in complexity, especially with regard to volatility in the financial market, has identified a new breed of programs that he categorizes as operating like genetic algorithms. This code has the ability to rewrite itself and Johnson claims that they're already in action on sites like Facebook. What are you doing?
We don't really know. What concerns Johnson and his colleagues is what conclusions the machines are drawing at a macro level. We can look at this stuff at a micro level. So it may be something straightforward, an algorithm that identifies your face in a photograph, but what's less clear is the set of conclusions that these genetic algorithms are drawing from the billions of interactions that they make, because those conclusions can be fed back into the system at the micro level.
So unless you understand the decision-making process that the program has made at the very top of that decision tree, it's really hard to identify and unpick the conclusions, or even identify how they might be skewing the system or changing the information within it. And these adaptive algorithms are becoming more widespread? Yes, no. Maybe experts like Johnson and the author, Cathy O'Neill, who wrote Weapons of Math Disruptions certainly think so.
There's this constant push and pull. So for example, companies like Amazon have to be on guard for bad actors who are trying to gain their pricing system. And the easiest way to do that for both parties is with an algorithm. So it's in the interests of both parties to have adaptive algorithms.
You block me, I change, I block you, you change, and the game carries on. And it can be fought at this really incredible speed. OK, so let's go back a step. Why the yes, the nor, the maybe?
Because nearly all of these algorithms are proprietary. Companies like Google don't want to publish the code because it makes it easier for developers to find ways to exploit the system. And of course, they don't want competitors cutting and pasting big lumps of it and building their own alternatives. But increasingly, there are calls for software companies, algorithms to be audited that countries should have bodies, whether they're government controlled or independent, that have some oversight.
Are we moving in that direction? Well, the EU introduced its data production laws earlier this year, but those probably don't go far enough. Interestingly, there's an increasing amount of bipartisan support in the US for regulation of the technology and software companies. President Trump has been issuing bombastic tweets targeting Google and Facebook and all the et cetera, over the past couple of weeks, alleging that the algorithms these companies use discriminate against conservatives.
At the same time, Democrats are moving towards proposals and legislation of their own, which are based on the interference at the last US elections. So you have the two sides moving towards a similar solution, albeit for very different reasons. Obviously, this is conjecture about how much difference would oversight make through this algorithm? Initially, it might have some limited impact.
I don't think it would have the kind of impact that either the Republicans and Democrats would hope for. Even in the most extreme scenarios, if some of these companies were broken up in smaller entities, other monoliths would probably rise from those companies. We use Facebook because everyone else is on Facebook, Ditto, Twitter, Instagram or Snapchat. We use Google as our search engine because it returns the best results.
Breaking those companies up is not going to change the nature of consumer behaviour. We gravitate towards what we consider to be the best solution that fits our lives. Good legislation to control the algorithms? Certainly, it could have some impact.
For example, in terms of creating a framework of responsibility, that would prevent companies from hiding behind secret code and claiming that they're not responsible for its effects. There was a case a while ago where a Toyota Camry appeared to accelerate for apparently no reason and the driver eventually ran off the road and died. It took nearly two years of investigations by software experts from NASA to figure out that there was this jumble of competing code, I think code is called its spaghetti code, and the algorithms that crashed against one another, resulting in weird and unpredictable effects, which in this case was this acceleration. And that was in an autonomous car?
No, that was in a normal car controlled by a human, but that's the level of software that's already in our systems. So you can only imagine what might happen in a self-driving car where there may be more than 100 million lines of code. Some of that code as well is designed to be dynamic and adapt. So if we go back to that Franken code article, Andrew Smith goes back to the arguments of George Dyson, who wrote about much of what we're experiencing more than 20 years ago in his book, Darwin and the Machines, we're building systems that are much too complicated for us to control, and the technology that controls them has to be at least as sophisticated as the systems themselves.
So in effect, we're building a world that relies on dumb algorithms. Would the government or third party oversight help to make the process safer? It's possible that it could even make it worse. Billions of lines of code making billions of interactions every day.
That's not the kind of thing that human beings can oversee. Realistically, we would have to build machines to check the algorithms, and those machines would have to be as sophisticated or more sophisticated than the algorithms that they're checking. Yeah, I've spotted the floor there. Yes, in order to police algorithms with intentions and logic processes that we can't comprehend, we have to build a bunch of algorithms with intentions and logic processes that we can't comprehend.
It seems like we have two choices. Either we step back from the technology or we race forward to create a more sentient and rational machines. Let's face it, there is probably zero chance of us going backwards and their son, calamitous disaster or economic crash, or maybe some meteorite impact forces us to do that. But we're still a long way from having those sentient machines.
Andrew Smith quotes Toby Walsh, an artificial intelligence professor at the University of New South Wales. And he points out that we have a lot of problems with creating AI that is truly capable of independent decisions. He questions whether we really will have millions of self-driving cars any time soon for one simple reason, that no one knows how to write a piece of code that will get a machine to recognize a stop sign. Isn't that one of the most basic requirements?
Go back to the Uber crash earlier this year. The software wasn't able to recognize what the woman was and predict her behavior based on that identity. Toby Walsh claims that we can't program a machine to recognize a stop sign. And that goes for everything from stop signs to translating languages to pretty much everything that we're looking at doing with artificial intelligence.
Yeah, but we don't have the ability to break down those processes into sufficient number of steps for a machine to understand? Exactly. So we design algorithms that allow the machine to learn around the issue instead. For example, the machine may analyze traffic flows and identify the fact that cars stop at an intersection and conclude there must be a stop sign here.
So it makes a note on its map when it gets there, it stops. But it isn't seeing or processing the information in the same way that we do. It's looking at data sets and it's making assumptions. So what's the solution?
You know, usually I do my best to try and leave you with, if not a silver lining, then at least a couple of, I don't know, shiny trinkets as a panacea. I don't know if I have anything to give away today. One of the experts that Andrew Smith quotes, a guy called Paul Wilmot, who specializes in quantitative analysis, he suggests that we all learn to shoot, make jam, and knit, which is, you know, it's a very matte-splained, post-apocalypse type solution. Neil Johnson, the Physicist and Financial Analyst, is a little bit more optimistic.
He says we may have to change the way we view the problem and adapt the way that we program algorithms accordingly. And will that work? As Johnson points out, we don't even have the scientific language to do that yet. At the moment, we program machines to create optimal outcomes.
What we're not addressing is what the worst possible outcome of those solutions could be. We have to find a way to predict that unpredictability. We have to find a way to calculate the likelihood that the collision of who knows how to do that. The question of who knows how many algorithms is going to create that unpredictable or worst outcome.
So we need a new science and we need it fast. Otherwise, that police robot is going to identify your breadsticks as batons, and it's going to put you down really hard. Anyways, Culture Box Matamatage, they're talking about killer codes. If you missed any parts of the show, you can download the podcast.
It's available on the BFM website and the BFM app. You can also check out culturepop.com. It's Culture with a K4 transcripts of this show. And of course, you can find out how to bring a little bit of Matt's Blaining and some killer codes to your company, the FM 89.9.
Thank you for listening to this podcast. To find more great interviews, go to bfm.my or find us on iTunes, bfm 89.9, the Business Station.