Longlisted for the National Book Award
New York Times Bestseller
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life — and threaten to rip apart our social fabric
We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated.
But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.
Tracing the arc of a person’s life, O’Neil exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort résumés, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health.
O’Neil calls on modelers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it’s up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.
— Longlist for National Book Award (Non-Fiction)
— Goodreads, semi-finalist for the 2016 Goodreads Choice Awards (Science and Technology)
— Kirkus, Best Books of 2016
— New York Times, 100 Notable Books of 2016 (Non-Fiction)
— The Guardian, Best Books of 2016
— WBUR's "On Point," Best Books of 2016: Staff Picks
— Boston Globe, Best Books of 2016, Non-Fiction
This taut and accessible volume, the stuff of technophobes' nightmares, explores the myriad ways in which large-scale data modeling has made the world a less just and equal place. O'Neil speaks from a place of authority on the subject: a Barnard professor turned Wall Street quant, she renounced the latter profession after the 2008 market collapse and decided to educate laypeople. Unlike some other recent books about data collection, hers is not hysterical; she offers more of a chilly wake-up call as she walks readers through the ways the "big data" industry has facilitated social ills such as skyrocketing college tuitions, policing based on racial profiling, and high unemployment rates in vulnerable communities. She also homes in on the ways these systems are frequently destructive even to the privileged: sloppy data-gathering companies misidentify people and flag them as criminals, and algorithms determine employee value during company-wide firings. The final chapter, in which O'Neil discusses Facebook's increasing electoral influence, feels eerily prescient. She offers no one easy solution, but has several reasonable suggestions as to how the future can be made more equitable and transparent for all.