How does The Spirit Level withstand a critic?

The success of Richard Wilkinson and Kate Pickett’s The Spirit Level (reviewed here in August) in setting the terms for much political discussion unsurprisingly triggered a burst of publications taking a sceptical look at their case. Prime amongst these is Policy Exchange’s publication Beware False Prophets, by Peter Saunders, whose title gives you a fair clue as to its line.

As the book says on its back cover:

In The Spirit Level, Richard Wilkinson and Kate Pickett claimed that egalitarian societies benefit rich and poor alike. Crime rates are lower, infant mortality is reduced, obesity is less prevalent, education standards are higher, average life expectancy is longer, social mobility is more extensive, and so on. They concluded that we would all benefit from a more egalitarian distribution of income…

[However] their evidence is weak, their analysis is superficial and most of the correlations in their book do not stand up.

So how does the Policy Exchange publication come to this conclusion and is its argument well founded? The critique is based on three core points: how strong the data is for the 23 countries that Wilkinson and Pickett looked at, what happens if you look at more countries and what the evidence is when comparing individual US states with each other.

On the 23 countries some individual criticisms are made of specific graphs that carry weight. However, the criticisms are not consistent with each other across the different graphs – sometimes Saunders suggests the data is overly dependent on one country, sometimes on others. Picking and choosing which countries to exclude from graph to graph does not make for a coherent case that there is no underlying pattern. Moreover, the basic point Wilkinson and Pickett are arguing is buttressed by numerous other studies beyond their own (see a partial round-up here) and so picking apart some of their individual data is a bit like picking apart the methodology of one individual pollster in some polls; it is a long way short of undermining the polling industry overall.

A more general point made is that there is a consistent difference between Scandinavia countries and Anglo countries. The Scandinavian countries are generally more equal and perform better on education standards, life expectancy and so on, whilst the Anglo countries are less equal and perform less well. That raises the question as to what extent it is the greater level of equality that makes Scandinavia different compared with other factors.

Peter Saunders also looks at what happens to the Wilkinson / Pickett data if you add in more countries, finding that in many cases the relationship between inequality and outcomes goes. However, he does this in part by adding in many small countries – those with populations of 1-3 million. So his critique really amounts to saying that the relationships may be different in small countries from large countries. The existence of such a difference is neither inherently implausible nor does it alter the political implications for the UK (population: a lot more than 3 million).

Saunders also adds in some large countries to the data set, though as he acknowledges these are generally poorer than the Wilkinson / Pickett set. As their theory is deliberately and explicitly one for affluent countries only, saying that something different happens in less affluent countries compared to more affluent ones does not undermine thesis. Their own response to Saunders’s work does not, though, directly address the question of how sensitive their results are to quite where you draw the cut off amongst countries nor why some of the affluent countries Saunders looks at were excluded from their study.

This mixed picture is common across all of Saunders’s areas of criticism – a few points that stick, some that simply sharpen the definition of the The Spirit Level‘s thesis as applying to medium to large rich countries, a few that Wilkinson and Pickett rebut and some where, even if you read their different accounts side by side (or look at what they said when appearing together, such as at an RSA meeting), they do not directly address each other’s points.

The ‘trump card’ that Wilkinson and Pickett have to play is the amount of other research that points in the same direction as their research. The two main weaknesses left are the question of whether there is really something else going on in Scandinavia and also (though not mentioned by Saunders) the point I made when reviewing The Spirit Level:

Not only does it rely overwhelmingly on comparisons across countries at the same point in time, rather than in tracking ailments varying over the years, the limited amount of such evidence deployed is almost all of the ‘inequality increased and then things got worse’ form. There is no automatic reason why, even if increasing inequality makes things worse, then decreasing it will make things better. The world is not always symmetrical. Moreover, even if the effect works strongly ‘in reverse’, is it the most cost-effective route to take? If inequality causes stress which causes social ills, is targeting stress going to be more successful?

My verdict then? Were The Spirit Level to have been original in the data patterns it found, these criticisms would add up to reasons to be cautious about its findings. However, because of the extensive other supporting data available, the findings still look solid even though – as I said in my review of it – the political policy implications are far more nuanced.

Amazon sell both The Spirit Level and Beware False Prophets.