"System Error is a triumph: an analysis of the critical challenges facing our digital society that is as accessible as it is sophisticated." — Anne-Marie Slaughter, CEO of New America
A forward-thinking manifesto from three Stanford professors—experts who have worked at ground zero of the tech revolution for decades—which reveals how big tech’s obsession with optimization and efficiency has sacrificed fundamental human values and outlines steps we can take to change course, renew our democracy, and save ourselves.
In no more than the blink of an eye, a naïve optimism about technology’s liberating potential has given way to a dystopian obsession with biased algorithms, surveillance capitalism, and job-displacing robots. Yet too few of us see any alternative to accepting the onward march of technology. We have simply accepted a technological future designed for us by technologists, the venture capitalists who fund them, and the politicians who give them free rein.
It doesn’t need to be this way.
System Error exposes the root of our current predicament: how big tech’s relentless focus on optimization is driving a future that reinforces discrimination, erodes privacy, displaces workers, and pollutes the information we get. This optimization mindset substitutes what companies care about for the values that we as a democratic society might choose to prioritize. Well-intentioned optimizers fail to measure all that is meaningful and, when their creative disruptions achieve great scale, they impose their values upon the rest of us.
Armed with an understanding of how technologists think and exercise their power, three Stanford professors—a philosopher working at the intersection of tech and ethics, a political scientist who served under Obama, and the director of the undergraduate Computer Science program at Stanford (also an early Google engineer)—reveal how we can hold that power to account.
Troubled by the values that permeate the university’s student body and its culture, they worked together to chart a new path forward, creating a popular course to transform how tomorrow’s technologists approach their profession. Now, as the dominance of big tech becomes an explosive societal conundrum, they share their provocative insights and concrete solutions to help everyone understand what is happening, what is at stake, and what we can do to control technology instead of letting it control us.
Philosopher Reich (Just Giving), computer scientist Sahami, and Stanford professor of political science Weinstein offer in this timely survey tips for how to "exercise our agency, reinvigorate our democracy, and direct the digital revolution to serve our best interests." Opening with the January 6 Capitol riots, the authors showcase the tension that marks society's relationship with technology social media was used to foment distrust in election results, and the leaders of Facebook and Twitter later leveraged immense power when they banned Donald Trump from their platforms. The authors explore major issues that they posit society needs to grapple with: the rise in the outsourcing of decision-making to algorithms, the immense amount of user data collected by tech companies, increasing automation, and the proliferation of hate speech and disinformation online. Their suggestions for how the country might better balance democracy and technology are evenhanded and nuanced: "A far more aggressive commitment to a right to data protection, alongside government agencies capable of enforcing that right, should be the first critical check on corporate power." Never falling into the trap of offering easy answers over deep analysis, this study is worth a look for readers worried about the outsize influence of technology on their lives and society.