BOOK REVIEW: Where’s the Noise?

This article was first published in the Quarter 4 2021 edition of Consider this. Click here to download the complete edition.

In Noise: A Flaw in Human Judgement by Nobel laureate Daniel Kahneman (also the award-winning author of Thinking Fast and Slow), Professor of Strategy and Business Policy at HEC Paris, Oliver Sibony and Cass R. Sunstein, Professor at Harvard Law School, the authors examine the sources and impacts of errors in judgement and look at ways to reduce them.

The authors define judgement as the process, as well as the product, of reaching a conclusion that can be summarized in a word or phrase. According to the authors, an error in judgement contains bias and noise. Examples of judgements can be performance reviews, bail applications and medical diagnoses.

The clearest analogy of bias versus noise I could find was not in this book, but an article from the Harvard Business Review¹. Consider a bathroom scale that consistently gives a reading that is too high, this is an example of bias. If the scale gives different readings when you step on it twice, the error is caused by noise, much like inconsistency. This is a good article to read if you are considering buying this book.

The book comprises six sections, each consisting of between 3 and 8 chapters.

Section one dives into how the criminal justice system can be noisy. I found the history on Judge Marvin Frankel’s crusade to reduce noise in this critical system to be a valuable lesson regarding the pros and cons of reducing noise. It provided a realworld example of attempts to reduce noise in a complex system, the challenges faced, the impact it has, as well as highlighting issues created by the attempts to reduce noise. It also introduces one to system noise, and the variability in judgements that should ideally be identical.

This is followed by an account of the authors’ first encounter with noise, when a consulting firm (where two of the authors worked) was asked by the executives of an insurance company to assess the potential value of increasing consistency in their policy and claims process. The authors labelled the analysis “detecting noise”, a noise audit. The result of the noise audit gave rise to a key theme throughout the book: wherever there is judgment, there is noise – and more of it than you think.

In section two the authors go into the details of measuring error. They define bias as the mean error (the average error occurring in numerous examples over time). Noise, meanwhile, is the variation around that mean, measured by one standard deviation. They then go about proving that noise and bias are both important components of error, and that total error will be reduced equally by noise or bias, if either is reduced by the same amount. I found large parts of this section tedious and unnecessary. A lot could have been summarised by some formulas in an appendix.

In the introduction, the authors suggest that if you are primarily interested in noise reduction, you should skip sections 3 and 4, and go straight to section 5. Section 3 goes into predictive judgements and explains how algorithms are free of noise and why machine learning is better at predictions than humans, given enough data.

Part 4 provides insight on the sources of noise and uses some of the theories developed in Kahneman’s Thinking Fast and Slow. In the above-mentioned sections, the authors also decompose error into bias and system noise, where system noise comprises level and pattern noise. Remember, judgement does not necessarily refer to the judiciary term – but it is used to understand the various types of noise in this example. Assume a judge is usually rather lenient in comparison with other judges, except when the defendant is a middle-aged man. Another judge is usually harsh, except when it comes to cases where the defendant is younger than 20. Level noise is the variability in the severity of judgements made by different judges. Pattern noise is how judges differ in which defendants require more severe or more lenient treatment.

Sections 5 and 6, as well as the appendices, are more practical, and I would recommend reading these to anyone looking for guidelines on how to detect noise within processes ranging from policy making to underwriting to judging wine. The reader is also introduced to several techniques to reduce noise, referred to as “decision hygiene”.

As investment managers, our team at M&G Investments must make frequent judgements (or decisions) that impact our clients’ wealth, making this an important topic for us. One key to successful investing for us is to eliminate as much bias and noise as possible in the investment process, and to be consistent around both level and pattern noise. While all humans make flawed judgements, we mitigate error and variability through employing a team approach and closely following a tried-and-tested process, among other tactics.

I must admit that after reading this book I am much more aware of noise, which I believe is what the authors set out to achieve. I did, however, find the book too long, and the structure sometimes confusing. It was not always clear why some topics were skimmed through, and others explained with tedious detail. I found myself re-reading chapters because I got lost in the detail, and at other times going off and reading Wikipedia pages on academic theories mentioned in the book, for instance Paul Meehl and Lewis Goldberg. Some consistency would have gone a long way in making it a more pleasant and engaging read. There is also not much attention given to the pitfalls that arise from findings like (quoting without context) “Noise is mostly a by-product of our uniqueness”.