Cathy O'Neil on Algorithms

Mo Lotman interviews mathematician Cathy O’Neil. O’Neil worked in the private sector to make the algorithmic systems that automatically judge and score us. Deeply troubled by what she saw, she went on to write the 2016 bestseller Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. She shares her insider’s look at how algorithms are gaming our world, with the worst consequences for those who can least afford them. Nobody escapes the grasp of algorithms in modern life. Per our recent article on Yelp, small business owners who have fallen afoul of Yelp’s manipulative algorithmic ranking have seen their businesses lose vast sums of money or be destroyed. If you live in the wrong neighborhood, algorithms may give you a worse credit rating, see that you spend more time in prison if you commit a crime, or even show you entirely different web pages when you are shopping online. O’Neil’s insights on algorithms are important in their own right, but algorithms are also the base layer of the AI that will be given more and more control of our society. What algorithms and AI have in common is the GIGO (Garbage In, Garbage Out) Problem: if you put in bad data, you get bad decisions. The other commonality is both are frequently “black boxes,” which is to say, even the people who create algorithms and AI often cannot explain the decisions they generate. O’Neil originally founded O'Neil Risk Consulting & Algorithmic Auditing to make algorithms fairer and more transparent, but has since expanded that to include assessing AI risks. (For further reading, here’s a 20-second primer from the CMS Wire article AI vs. Algorithms what’s the difference?“An algorithm is a set of instructions — a preset, rigid, coded recipe that gets executed when it encounters a trigger. AI on the other hand — which is an extremely broad term covering a myriad of AI specializations and subsets — is a group of algorithms that can modify its algorithms and create new algorithms in response to learned inputs and data as opposed to relying solely on the inputs it was designed to recognize as triggers. This ability to change, adapt and grow based on new data, is described as ‘intelligence.’ ”) Get full access to The Technoskeptic Magazine at technoskeptic.substack.com/subscribe