Review of Weapons of Math Destruction

I finished reading Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil.📚 Algorithms surround our daily lives.

In this Book author, Cathy O’Neil tells us how they work and why they are bad for humankind. We visit the algorithm governing the juristic systems and schools. And also, look at how they control our daily lives via deciding what we pay for a loan.

Discussion

Please note that from here on out, the text can contain spoilers of the book.

I liked reading this book, and it covers a topic I find pretty exciting and worrying for our society. The book is very US-centric, but I started searching for answers to these topics for Switzerland now.

The book starts with an explanation of what a model is and gives examples for this. A model is always a simplification of the real problem, and errors are to be expected. The model contains the choices and biases of its creators. In the end, models are opinions encoded in mathematics.

To rate a WMD, the author proposes to analyze them regarding the following three dimensions: opacity, scale, and damage. The most important question is, do you know the results when the model is applied to your situations?

When creating a model, you need to be careful which variables you choose. These variables have a lot of influence on the lives of people. An example of this is with the police patrol allocation system. There is a big difference if you include nuisance crimes (drinking in public, drug use) compared with only tracking “real” crimes (murder, homicide, rape). When you include the first group, the cops have more to do because these crimes are usually not planned and primarily committed in the same region. So it causes more police to be in these neighborhoods, and then they find more crime again. And one big caveat is that these nuisance crimes happen for the most part in the poorer parts of the city – confirming the bias that poor people are inferior.

To change the power of WMD, we need to change our laws and make the owner of such systems follow defined rules. And perhaps also miss out on some efficiency on the way. Data scientists made some attempts at self-regulation like a Hippocratic oath equivalent and Maths and tech specialists need Hippocratic oath, says academic. But this is only part of the solution because data scientists are replaceable as long as there are no rules for the owners of WMDs.

Another worrying chapter in the book looked at the experiment Facebook did some time ago. Facebook managed to change voter turnout by around 300k – a result that sounds good at first glance. But some ballots are decided with less than 300k differences in votes, and when you now control a tool that can generate such differences, you can influence the results.

We need to be very careful with such technology. And perhaps the state should also regulate it. The chapter reminded me of the book Zucked: Waking Up to the Facebook Catastrophe (link to my review of the book). It looks at this topic with more details.

In the end, WMD creates an uneven playing field and is seldom updated again as most of the soft criteriums we humans use to evaluate something are very hard to model. This book also reminded me of Hello World - How to be Human in the Age of the Machine (link to my review), which gives a good overview of the different types of WMDs.

• /review
• /wmd
• /algorithms
• /model