Computer algorithms now shape our world in profound and mostly invisible ways. They predict if we’ll be valuable customers and whether we’re likely to repay a loan. They filter what we see on social media, sort through resumes, and evaluate job performance. They inform prison sentences and monitor our health. Most of these algorithms have been created with good intentions. The goal is to replace subjective judgments with objective measurements. But it doesn’t always work out like that.
“I don’t think mathematical models are inherently evil — I think it’s the way they’re used that are evil,” says mathematician Cathy O’Neil, author of the book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. She has studied number theory, worked as a data scientist at start-ups, and built predictive algorithms for various private enterprises. Through her work, she’s become critical about the influence of poorly-designed algorithms.
The Age of the Algorithm