The Spirit Level has been debunked. More or less.

The BBC Radio 4 show More or Less looks at the facts behind well publicised statistics. The show is presented by the author of The Undercover Economist, Tim Harford and, as a regular listener, I was intrigued to hear that the first show of the new series promised to 'decode The Spirit Level debate'. For a show dedicated to debunking junk statistics it was obvious subject matter, but I wondered how Harford could 'decode' such a voluminous topic over the airwaves in one show.

In the end, he didn't need to. From the outset, Harford admitted there were too many competing claims to fit into a magazine show, and instead interviewed The Spirit Level's co-author Kate Pickett, who did more damage to the reputation of The Spirit Level in the space of ten minutes than any number of supposed "idea wreckers".

The interview (listen here for the next couple of days, or subscribe to the podcast) is essential listening if you've been following the controversy, as Pickett struggles to answer some fair and simple questions. Strangely enough, Wilkinson and Pickett's Equality Trust website—which is normally so quick to let people know when The Spirit Level has been mentioned in the media—have yet to post a link to this interview. And since the audio file won't be available for much longer, I've transcribed some of the key moments for posterity...


Falling back on other people's research

Wilkinson and Pickett's first line of defence is to claim that there are 100s of peer-reviewed studies which support their conclusion. As I have said before, this is just not true. Most of the studies they reference in The Spirit Level do not even mention income inequality.

In the More or Less interview, Kate Pickett once again claimed there was a "vast body of research" behind The Spirit Level. Tim Harford picked her up on it...

KP: We wrote a book that's intended to be a synthesis of a very vast body of research. Not only our own, but those of other people... There is a consistent and robust and large body of evidence showing the same relationship.

TH: That's an interesting point that you make. Often, in response to critics, you have referred not to your own book, not to your own data but to other published research. I'd really like to focus on the research that's presented in your book. It's very easy to say there are 50 papers, there are 200 papers, that support our research but we don't really know how you've selected those papers.

KP: We actually have completed a systematic review of all of the studies of income inequality and health, and we reference that in our book. We do examine things systematically and certainly—when we are doing our own research, publishing in peer-reviewed journals—we have to be aware of all the literature in the field. But that doesn't mean that every paper in the field has good methods, comes to the right conclusion, studies the right thing.

TH: I absolutely agree. One of the papers that you refer to in support of your argument on women's empowerment and women's status which was published in 1999 by Kawachi and some other authors, you claim supports your findings on women's status and income inequality. I've looked at their abstract. It doesn't seem to attack that question at all. It's simply on another subject—a somewhat related subject but not on the subject of income inequality.

KP: They've definitely published and we may have inadvertently put the wrong reference into that document [laughing nervously]. But Kawachi and Kennedy have certainly published finding a relationship between income inequality and women's status. The paper is Women's Status and the Health of Women and Men: a view from the States and it was published in Social Science and Medicine in 1999.

TH: That's the one I'm looking at.

Note: On page 58 of The Spirit Level, it states: "Researchers at Harvard University showed that women's status was linked to state-level income inequality. (36)"

Reference 36 is the Kawachi study ('Women's status and the health of women and men: a view from the States', 1999). As its title suggests, this study compared women's status with health, not with inequality. Indeed, the authors found a correlation between women's status and health even after controlling for income inequality.

Failure to look at other variables

The Spirit Level relies on the conceit that countries are fundamentally the same, with income inequality being the main variable that distinguishes them. This allows Wilkinson and Pickett to disregard other variables such as income, culture, history, demography, ethnicity, geography, law, politics and climate. Ignoring other variables and confounding factors would be a flaw in any study—as Harford points out, it breaks a basic rule of epidemiology—but when entire countries are being studied, this flaw becomes overwhelming. Pickett's response is revealing: she and Wilkinson do not "believe" that factors other than income inequality have an effect on a country's performance, so they don't bother looking at them.

TH: All of your studies are what are called bivariate analysis. In other words, they're all income inequality plotted against some other variable. Now, my understanding of best practice in social sciences is that you would always control for other variables. You would include 2, 3, 4, 5, 6 other variables and...

KP: Well, you wouldn't do that arbitrarily. You would do that if you believed those variables were potential alternative explanations of the relationship you're looking at.

TH: So, if I understand your statement correctly, you didn't include any multiple variable analysis because you just think that actually none of these variables are of interest—none of them are potential alternative explanations and you can just do the simple income inequality versus x analyses?

KP: That's right, but of course, again, other researchers have conducted studies that do control for more, where, as well as examining the effect of income inequality at the level of the whole society, people include individual's own levels of income or levels of education in those analyses and, again, those bear out our findings in relation to health.

TH: We come again to...you're basically rowing back from your analysis and saying...

KP: No. Indeed I'm not...

TH: "Don't look at our analysis, look at these other people because they support us."

KP: We believe that to control for individual income is actually over-controlling, so we would not consider that best practice.

Academic criticism

Although well received by some journalists and politicians, The Spirit Level has received a much cooler reaction from academics. One of the few serious academics to have reviewed the book was John Kay, Professor of Economics at London Business School and former Director of Institute of Fiscal Studies. Pickett's response to Kay's review speaks volumes...

TH: When John Kay reviewed your book in the Financial Times —and I believe John Kay would be broadly sympathetic to your idea that egalitarianism is important—he wrote: "The evidence presented in the book is mostly a series of scatter diagrams with a regression line drawn through them. No data is provided on the estimated equations, or on relevant statistical tests. If you remove the bold lines from the diagram, the pattern of points mostly looks random, and the data dominated by a few outliers." Do you think that's fair?

KP: No, I don't think it's fair. [Testily] He didn't read the book thoroughly, obviously.

Outlandish claims 

In the last chapter of The Spirit Level, Wilkinson and Pickett make some extraordinary claims about what could happen if Britain reduced income inequality to Scandinavian levels. These include: teen births falling to a third of current rates, mental illness being halved, life expectancy rising by a year and the murder rate falling by three-quarters. Harford asks her about the last of these predictions.

TH: Clearly your book is a systematical analysis and partly also a political book. You have a political case to make—there's nothing wrong with that. You have public policy actions that you would like to see taken. But do you think you may have overstated some of those? Let me give you an example. On page 268 of you book—towards the conclusion—you say that if Britain became as equal as Japan, Norway, Sweden and Finland, homicide rates could fall by 75%. But as I'm sure you've had pointed out to you by now, the UK's homicide rate is already below the average of those four countries.

KP: It's not actually. It's been pointed out that it's below Sweden. It's not below the average of those countries. Those claims [ie. in The Spirit Level], they're based on regression models and of course they're only as good as they're model they're based on.

TH: [Incredulously] But.. sorry... but you've made that claim!

KP: Yes, yes, we do...

TH: And you stand by it?

KP: Yes. That Britain would become a much healthier and more socially better functioning place if it were more equal.

TH: You said that if Britain became as equal as these four countries, homicide rates could fall by 75%. Do you not feel that's really overstating the case, or do you stand by that?

KP: That's based on the model. I mean, I think we could try it and see.

[end of interview]

TH: Kate Pickett, co-author of The Spirit Level. We did go to her Equality Trust website, by the way, and downloaded the data on homicide rates in the UK and in the relevant four countries and it does seem that I was right to say that the UK's homicide rate is already below the average of those four countries. You're listening to More or Less...

Today's report in The Guardian

Today's report in The Guardian will no doubt draw the usual hate and bile from people who have no intention of reading my book. Still, a few quick points...

Nobody asked me or paid me to write this book. I never set out write a critique of another book. While I was researching a completely different topic, I bought and read The Spirit Level because, as I said to The Guardian, it was "influential and informing debate." Those are the kinds of books I like to read, whether from left or right. When I started fact-checking The Spirit Level I realised that it was too big a subject to squeeze into an article or blog post and The Spirit Level Delusion was born.

When Dr Patrick Basham kindly offered to write the preface for the book, I published it in association with the Democracy Institute, of which he is the director. Had I known this would leave the book open to accusations of being written by a "wrecker" from a "rightwing thinktank" I wouldn't have bothered. You live and learn.

I knew when I wrote it that the dogmatic right wouldn't be interested because they wouldn't have read The Spirit Level. I knew the dogmatic left wouldn't be interested because they'd put their fingers in their ears if anyone raised difficult questions about such a politically useful text. But I also knew that there would be some people in between who had enquiring minds and a genuine interest in the issues. Perhaps I overestimated how many fell into that camp.

The Guardian quoted a few words from a twenty minute interview. No complaints, that's the way it goes...

He [Snowdon] does not believe that The Spirit Level's claim that the psychological effects on society of income inequality are so great to cause widespread social ills. "I don't think people outside the intelligensia worry about inequality," Snowdon said. "The working class don't worry about how much Wayne Rooney is earning."

It's a crude example, but it serves to illustrate one of the fundamental problems with The Spirit Level. It cannot be stressed often enough that Wilkinson and Pickett's hypothesis rests on the psychological (or 'psychosocial') effects of living in a less equal society, not the material effects of poverty.

When people say that they find The Spirit Level's conclusions to be 'intuitively' true, or that they appeal to 'common sense', I wonder whether they fully appreciate that Wilkinson and Pickett are not blaming poverty, low income or low living standards per se. They are talking about something much less tangible—a sense, a feeling, a response—to other people's wealth. As someone who happens to be in the bottom 20% of earners myself, I don't personally feel traumatised by the existence of the super-rich. Perhaps that's just me, but there is also very little empirical evidence that the psychological response to inequality has a significant effect on people's day-to-day lives.

Wilkinson and Pickett would disagree, but the (left-wing) economist JK Galbraith understood this back in 1958 when he wrote The Affluent Society:

Envy almost certainly operates efficiently only as regards near neighbours. It’s not directed towards the distant rich.

In a later preface to The Affluent Society, Galbraith returned to the issue of inequality, making it clear that so long as people's own living standards were improving, they are not troubled by the thought of other people becoming still richer:

When, as suggested in this book, men and women are employed and at continuously improving wages or salaries, they are not greatly concerned that others, with whatever justification or absence of justification, have more, even greatly more.

More recently, in Status Syndrome, the (left-wing) epidemiologist, Michael Marmot discussed the stubborn refusal of ordinary Americans to become less happy even as their country became less equal. He made a telling comment about who is really 'stressed' by income inequality:

Changes in income inequality did not affect happiness levels of the poor. The subgroup of the population whose happiness declined when income inequality increased, were richer people who described themselves as on the left politically.

I discuss this issue in more detail in the later chapters of the book.

No doubt there is resentment at some of the grotesque disparities of wealth that exist (and have always existed), but that resentment would have to be truly monumental for it to be the main driver of an entire country's performance across so many criteria. Very few variables—let alone psychological variables—show up in aggregate data from whole nations. The psychosocial effect of income inequality is not one of them, and Wilkinson and Pickett have to perform all sorts of twists and turns to make their case to the contrary. At best, The Spirit Level gives a cock-eyed view of the way the world is.

The case for greater income equality remains an ethical, moral and political issue. It cannot be 'proved' by social science.


My response to Wilkinson and Pickett's answers to my 20 Questions is here .

Some of the graphs from The Spirit Level Delusion are here.

Links to other sites discussing the debate over The Spirit Level can be found on the right-hand side of the page.

Reply to Prospect magazine article

This is my reply to an article in Prospect by Richard Wilkinson and Kate Pickett. That article can be read here. The article that inspired it is here.


I’m pleased to be able to agree with Wilkinson and Pickett (W & P) on one point. I’m not a public health researcher and was surprised to be described as such. That, sadly, is where agreement ends.

Wilkinson and Pickett once again imply that they are merely the messengers of a scientific consensus and that there are 100s of peer-reviewed studies saying the same thing. Can we please put this one to bed? Even if quantity was a substitute for quality, the argument does not apply here. There is a large body of conflicting research about health and inequality and a smaller body of research studying violence and inequality. Both are hotly debated, not least because it is very difficult to isolate the effects of income inequality from the effects of low income.

Beyond this, Wilkinson and Pickett are out on their own, making claims that have virtually no support in the scientific literature. In contrast to what he says in the third paragraph of this rejoinder, Wilkinson recently told the magazine International Socialism:

"There are about 200 papers on health and inequality in lots of different settings, probably 40 or 50 looking at violence in relation to inequality, and very few looking at any of the other things in relation to inequality. In a way, the new work in the book is all these other variables—teenage births, mental illness, prison populations and so on—and the major contribution is bringing all of that into a picture that had previously been just health and violence."

W & P confuse making assumptions based on other people’s research with having those people actually agree with them. For example, they cite studies that quite reasonably associate overeating with stress, but it does not follow that obesity rates vary internationally because the population is stressed about inequality. At best, this is speculation.

W & P continue to cite perfectly sound studies showing there to be social gradients to health and social problems as evidence that inequality affects a nation’s overall performance. It does not. These are completely different issues.

Which leaves us with W & P’s own evidence, which relies on comparing whole countries, a notoriously unreliable method which allows unlimited scope for misinterpretation. The criticisms recently made of this evidence by myself and others closely echo criticisms made in peer-reviewed journals when Wilkinson used similar methods in the past. They also echo criticisms made by the few serious academics who have reviewed The Spirit Level.

Anyone who believes that W & P “never pick and choose data points to suit our argument” should compare references 2 and 6 in The Spirit Level (p. 271) and ask themselves why one year’s data were used for one graph and another year’s data used for the next. Anyone who believes that they use “the same measures of inequality” should turn to page 224 and ask why a dramatically different measure of inequality was preferred when working hours were studied (clue: see how it looks when we use W & P’s more usual measure of inequality). Anyone believing that they have not “picked problems to suit our argument” might ask why they show how much overseas aid is given by a country’s government, but do not show how much is given privately (there is no correlation with inequality when the two are combined).

As for always using the same group of countries, one of The Spirit Level’s most serious flaws is the baffling assumption that “rich market societies” come in batches of 50. If there is to be a cut-off point beyond which economic growth has “largely finished its work”, it should be based on something more than a round number. Without a convincing justification for why places like the Czech Republic and South Korea – let alone Hong Kong – cannot be considered rich market societies, we must ask the next question: why do these societies conspicuously fail to fit Wilkinson and Pickett’s theory? The United Nations classes these countries as being of “very high human development”, why doesn’t The Spirit Level?

I hope that readers will take the time to look at these issues themselves, but, if not, they should at least take a deep breath and ask themselves which is more plausible – a theory that seeks to explain the workings of whole societies by reference to a single factor, or one that says that a country’s performance is the result of countless historical, geographical, political, legal, demographic and economic factors, of which the public’s response to income inequality may or may not be one.

Are people in 'more equal' countries more likely to vote?

In the latest edition of The Spirit Level, Wilkinson and Pickett put forward the idea that voter turn-out is higher in "more equal" countries.

There is evidence from societies where voting is not compulsory (as it is for instance in Australia) that voter turn-out may be lower in more unequal countries. Whether or not this reflects a greater separation of interests and an increasing sense of 'us and them' between people at opposite ends of the social ladder, it certainly suggests that too much inequality is a threat to democracy.

The Spirit Level, p. 295 (revised edition)

The reference given to support this claim is cited as 'B. Geysa, 'Explaining voter turnout: A review of aggregate-level research'. This study is available online. Aside from getting the name of the author wrong (it's Geys, not Geysa), Wilkinson and Pickett totally misrepresent the study's findings. The word 'inequality' appears just three times in this 27 page review and although it briefly addresses the question of whether inequality might reduce voting turn-out, it clearly concludes that most studies have shown that it doesn't. At no point does it even vaguely imply that "voter turn-out may be lower in more unequal countries", let alone that this represents a "threat to democracy".

Wilkinson and Pickett do not present a graph of their own as evidence, but thanks to recently published OECD data on voter turn-out, we can put the hypothesis to the test. The graph below shows all rich OECD countries, excluding tax havens (as per The Spirit Level) and Australia (where voting is compulsory).

It would be hard to produce a straighter line. Even with a spirit level.

Fact-checking The Spirit Level debate

Thanks to all who organised and attended the debate at the Royal Society of Arts on 22nd July. Due to the structure of the event, Peter Saunders and myself did not get the chance to reply to Wilkinson and Pickett's presentation but, although I haven't yet listened back to the debate (I'll post the video link when it is available), there are a few basic factual errors that need clearing up...


1. Trust

In response to my claim that the correlation between trust and inequality depends entirely on the Scandinavian countries, Pickett presented a graph which showed the same data (from The Spirit Level) but with the Scandinavian countries excluded. A correlation remained, albeit weaker.

This is true, but the two critical problems with their graph on trust remain: (1) As with all Spirit Level graphs, it excludes several wealthy countries; (2) it relies on data from the 1990s which has been superseded by the 2000s data (which is used in The Spirit Level Delusion). When the most recent data is used there is clearly no correlation between trust and inequality.

Source: World Values Survey


2. Happiness

Richard Wilkinson dismissed the evidence showing that happiness is not correlated with income inequality—but is (positively) correlated with income—by saying that happiness does not have a social gradient.

This is not true. Happiness certainly does have a social gradient. One of the best known demonstrations of this can be found in a paper by Robert Easterlin from 1974. It clearly shows happiness rising in line with income.

This particular article gave rise to the so-called 'Easterlin Paradox' and is one of the most famous papers in economics. It is certainly the most famous study in the field of 'happiness studies', and as such it is hard to believe that Wilkinson can be unaware of it.


Source: Does Economic Growth Improve the Human Lot? Some Empirical Evidence, Robert Easterlin, 1974



3. Health

In response to our evidence showing no relationship between inequality and life expectancy, Pickett referred to a 2009 study from the British Medical Journal. Wilkinson and Pickett also cited this study in their response to Peter Saunders and in their response to a recent critique I co-authored in the Wall Street Journal. In the latter, they said that the BMJ study "shows unequivocally that inequality is related to significantly higher mortality rates."

In fact, the BMJ study concludes that:

The results suggest a modest adverse effect of income inequality on health, although the population impact might be larger if the association is truly causal... The findings need to be interpreted with caution given the heterogeneity between studies.

Unequivocal?



4. The Spirit Level was written in 2007?

Kate Pickett referred to one of my 20 Questions, which reads:

Why do you say that the USA’s decline in homicide ended in 2005 when 2008 saw the lowest number of homicides since 1965? As you must know, America's murder rate has halved in the last two decades despite rising inequality.

All of which is true. Wilkinson and Pickett claim that the US homicide rate "started to rise again" in 2005 (p. 142). In fact, the murder rate fell in 2007 and 2008 and is now at its lowest rate since 1965.

During the debate, Pickett explained that there was a simple reason for them ignoring the ongoing decline in the US homicide rate—their book was written in 2007! That got a good laugh, but it is not true. As can be seen from the references at the end of The Spirit Level, they were still writing—and finding new sources—well into 2008.

For example:

(95) S. Bezruchkra et al., 'Income economic equality and health: the case of postwar Japan', American Journal of Public Health, (February 2008)

(298) K. Pickett & R. Wilkinson, 'People like us: ethnic group density on health', Ethnicity and Health, (September 2008)

(379) W. Hutton, 'Let's get rid of our silly fears of public ownership', The Observer, (April 2008)

Clearly there was time to acknowledge the US homicide rate in 2007, if not 2008.


5. Inequality in Japan

During the Q & A session, I mentioned that there are questions over how equal the distribution of wealth in Japan really is. I pointed out that Gini figures from the OECD show Japan to be on a par with Spain and Portugal (this was a mistake on my part—I meant to say Spain and Greece). Pickett responded by saying that these figures are for Gini before tax.

This, again, is not true. The OECD provides inequality data both before and after tax. The most recent data, from the mid-2000s, show:

Japan (before tax): 0.44

Japan (after tax): 0.32

Spain (before tax): 0.41

Spain (after tax): 0.32

Greece (before tax): 0.43

Greece (after tax): 0.32

Sweden (before tax): 0.43

Sweden (after tax): 0.23

As the OECD states in a 2006 article:

The Gini coefficient measure has risen significantly since the mid 1980s from well below to slightly above the OECD average and the rate of relative poverty in Japan is now one of the highest in the OECD area.

I have no firm view on which of the two sources (UN or OECD) have the most realistic figure for Japan, but as a point of fact the OECD figures I mentioned were after tax.

Source: OECD

The Spirit Level, the Policy Exchange and the race card

It was enough to suggest a vast right-wing conspiracy. Last week, three debunkings of the left’s new favourite text The Spirit Level appeared in as many days—first from the TaxPayers’ Alliance, then from Policy Exchange and then in the Wall Street Journal. This coincidence (I assure you it was a coincidence) was enough to rouse The Spirit Level’s authors—social epidemiologists Richard Wilkinson and Kate Pickett—into responding, to the Policy Exchange’s rebuttal at least. Disappointingly, this response was heavy on name calling and light on substantive arguments.

Wilkinson and Pickett insisted that their book was the result of “decades of research”. True enough. Richard Wilkinson has doggedly pursued the theory that “unequal societies are almost always unhealthy societies” for the last 35 years. During that time, life expectancy has risen rapidly, despite growing inequality, leaving ultra-egalitarian Denmark with the lowest life expectancy of any country studied in The Spirit Level. That none of this has swayed Wilkinson from his hypothesis is a tribute to his indefatigability, but stamina alone is not enough. People have spent their lives on more quixotic endeavours than Wilkinson, but that should not necessarily recommend them to us.

Wilkinson and Pickett stress that many peer-reviewed articles have offered at least partial support the relative income hypothesis. This is also true—albeit only in the area of health—even if Wilkinson has written a large number of them himself, but there are also plenty of peer-reviewed articles that beg to differ. Hence the long-running academic debate about inequality which The Spirit Level has done much to popularise but little to resolve.

Amongst that large body of scientific literature, there have been several suggestions of selection bias on Wilkinson’s part (that’s ‘cherry-picking’ to you and me) which have been echoed recently with regards to the The Spirit Level. Wilkinson and Pickett throw the same accusation at Peter Saunders, the author of the Policy Exchange critique, saying that he selectively removed a number of countries from his analysis. This would be a potent criticism had Saunders picked the most useful subset of countries and used them throughout. In fact, he only occasionally excludes a handful of outliers to show that Wilkinson and Pickett’s regression lines are being dragged this way and that by a few special cases, thereby creating the illusion of a sloping gradient where none exists.

It is valid, indeed crucial, to demonstrate this point. Take their graph on obesity, for example. Even the casual reader can see that the line goes upwards as a result of fat Americans and skinny Japanese. There is a conspicuous lack of a dose-response relationship when it comes to the other countries, but note the absence of Singapore and Hong Kong in this graph, both wealthy societies which marry extreme inequality with low rates of obesity (and low rates of most other health and social problems). Wilkinson and Pickett are no strangers to “arbitrarily cutting out certain countries” themselves.

Conversely, Wilkinson and Pickett accuse Saunders of including too many countries (he shows 44 to Wilkinson and Pickett’s 23). This is a view I have a little more sympathy with. Places like Russia and Chile clearly have absolute poverty in a way that France and Norway do not. Since Wilkinson and Pickett accept that wealth improves countries up to a point, it could certainly be said that several of the nations included in Saunder’s analysis have not reached that point. However, as both The Spirit Level Delusion and the TPA report show, one need only to include countries which are as wealthy or wealthier than Portugal to show that there is no relationship between inequality and most health and social problems. Wilkinson and Pickett have yet to justify their decision to exclude places like South Korea, the Czech Republic, Slovenia and Hong Kong.

Wilkinson and Pickett then aim a blow below the midriff when they accuse the Policy Exchange of being “from the political far right” and accuse Saunders of a “seriously racist slur” for showing the tendency of US states with large African-American populations to have the most social and health problems. The correlation with ethnicity is strong—much stronger than the correlation with inequality—but Wilkinson and Pickett dismiss it as “racist because it implies the problem is inherently the people themselves rather than their socioeconomic position”.

If Wilkinson and Pickett think it is racist to say that there are a host of cultural and historical reasons why blacks tend to do worse than whites in the USA, then there are plenty of black community leaders and black politicians who are racist. No serious discussion of modern-day America can ignore the legacy of slavery and segregation, as well as the more subtle forms of ongoing discrimination which continue to hold African-Americans back. There is no doubt that these factors contribute to income inequality, but to say they are caused by inequality is highly questionable.

The reasons why the black homicide rate is much higher, and black life expectancy much lower, than the corresponding rates for white Americans are many and varied but in The Spirit Level, they are—as ever—reduced to symptoms of inequality. This will not do. Income, inequality and ethnicity are so closely intertwined in the United States that it is difficult to see where one stops and the other starts, but Saunders argues persuasively that inequality is not the main driver.

A significant clue lies in the pages of The Spirit Level itself. Wilkinson and Pickett note with surprise that they can find no association between inequality and poor mental health (p68-69) and then mention, almost in passing, that rates of mental illness are evenly distributed between different races. This should have been a Eureka moment but, as Saunders writes, “they fail to draw the obvious conclusion from their failure to find a relationship with inequality, which is that they only get state-level correlations with income inequality when there are underlying correlations with race to generate them” [emphasis in original]. Inequality is a symptom, not the cause.

This highlights one of the main problems with The Spirit Level. The myopic obsession with income inequality blinds the eye to the countless cultural, political, historical and demographic reasons why countries are as they are. Wilkinson and Pickett’s hypothesis requires one to believe that these are all rooted in inequality, but taking each in turn we can see how implausible that is. Their ‘theory of everything’, like all grand unifying theories, makes an extraordinary claim and requires extraordinary evidence. As was repeatedly demonstrated last week, the evidence provided in The Spirit Level withers under the light of serious scrutiny. Crying ‘racist’ will not make it more robust.

[This article was first published by the TPA on 16.07.10]


[Peter Saunders has responded to Wilkinson and Pickett's attack on his website]


[Tino Sanandaji—one of the authors of the WSJ article—makes some additional comments on his blog]

Correlation coefficients and p-values

The original edition of The Spirit Level did not show correlation coefficients or p-values for any of the graphs, but when I wrote The Spirit Level Delusion, I included r-squared coefficients to show the strength of the associations.

Later editions of The Spirit Level include an appendix which shows Pearson Correlation Coefficients and two-sided p-values. As the Pearson coefficient shows a different figure than the r-squared, I have included the Pearson coefficients and the two-sided p-values for The Spirit Level Delusion graphs below for anybody who should want to compare them.

The first number is the correlation coefficient (the strength of the correlation). The figure in brackets is the p-value (the confidence that the correlation has not been caused by chance). Both numbers go from 0.00 to 1.00. For the first figure, the higher the number, the stronger the correlation. For the latter, lower numbers represent stronger associations. Statistical significance is usually represented by a p-value of at least 0.05, with 0.01 or less being preferred.

Life expectancy (UN 2004): -0.4515 (0.03)

Life expectancy (UN 2006): 0.16 (0.42)

Life expectancy (UN 2009): 0.22 (0.27)

Obesity: 0.01 (0.96)

Smoking: -0.31 (0.11)

Alcohol: -0.25 (0.21)

Infant mortality: 0.13 (0.51)

Trust: -0.29 (0.13)

Teen births: 0.32 (0.10)

Happiness: 0.05 (0.81)

Homicide: 0.30 (0.12)

Recycling/suicide: 0.64 (0.04)

Prison: 0.59 (<0.01)

Crime: -0.15 (0.45)

Victim of crime: -0.344 (0.21)

Recycling: -0.739 (<0.01)

Charity: 0.40 (0.25)

Foreign aid: 0.527 (0.01)

Single parents: -0.06 (0.285)

Suicide: -0.475 (0.01)

Unemployment: 0.033 (0.88)

Community life: 0.54 (0.03)

Alcohol/divorce: 0.35 (0.12)

Education: -0.154 (0.44)

Quality of life: 0.13 (0.51)

Homicide/suicide: -0.03 (0.86)

% GDP in tax/inequality: -0.65 (<0.01)

Happiness/income: 0.62 (<0.01)

Trust/income: 0.56 (<0.01)

Overseas Aid

[This footnote relates to Chapter 6 of The Spirit Level Delusion]


Since The Spirit Level Delusion was published, I have come across the Index of Global Philanthropy and Remittances (2009) which provides overseas aid figures for 22 countries. Crucially, this index combines state aid with private donations. Shown as a percentage of gross national income, it is clear that more equal countries are neither more nor less generous to the developing world than the less equal countries. Wilkinson and Pickett are only able to argue otherwise by ignoring all private donations.

Case study: Life expectancy

One of the central claims of The Spirit Level is that 'less equal' countries have lower life expectancies than more egalitarian countries. Its authors claim that the psychological stress of living in a less egalitarian society affects the health of all—rich and poor—and that this manifests itself in lower life expectancies. 

To demonstrate this they show a graph that looks very much like this:

Inequality is shown on the horizontal axis and is the difference between the richest 20% and the poorest 20% (eg. in Sweden, the richest 20% are 4 times wealthier than the poorest 20%). As you can see from the graph, there seems to be a downward trend in life expectancy from the more equal countries, especially Sweden and Japan, to the less equal countries. The low life expectancies of Denmark and Finland should make us wonder whether this graph really proves that egalitarianism results in good health, but a broad correlation remains nonetheless.

But there are two major problems. Firstly, a number of wealthy societies are missing from this graph. In particular, where are Hong Kong, South Korea, the Czech Republic, Slovenia? All of them are wealthier than Portugal and should be shown. 

Secondly, there are questions over the data used to ascertain life expectancy. Wilkinson and Pickett use figures from the United Nations Human Development Report, a respected and accurate source, but they use data from the 2004 edition. This is an odd choice because elsewhere in The Spirit Level, the authors rely on the 2006 edition. They even use life expectancy figures from the 2006 report elsewhere (the graph on page 7) so we know they were aware of them. 

This is their reference for a graph they use to show life expectancy earlier in the book...



But this is their reference for their graph showing life expectancy against inequality...

So why the use of old statistics and a carefully selected sample group? If we look at the data from the UN's 2006 report—and include all relevant countries—a very different picture emerges.

The association between inequality and life expectancy has disappeared, replaced by a modest trend in the opposite direction. And to prove that the 2006 report is not an anomaly, here are the results from the 2009 report:

Again, there is no correlation. Indeed the three worst performing countries have a very equal distribution of wealth (Czech Republic, Slovenia and Denmark). 

It's worth noting that the inequality/life expectancy hypothesis is not new. It first came to prominence in a 1992 British Medical Journal article, written by none other than Richard Wilkinson. That article inspired a flurry of research and, in The Spirit Level, the authors refer to a “vast literature” on the subject. There is, however, no mention of how much of this vast literature was written by Wilkinson himself, nor that much of the rest was critical of his theory. 

His BMJ study was debunked at length in the same journal in 1995 by Ken Judge. Judge pointed out numerous errors in Wilkinson’s research, including the use of “inappropriate” data. He criticised Wilkinson for using the lowest 70% of families as a measure of inequality when a more conventional measure is the bottom 10% or 20% of individuals. “The suspicion,” wrote Judge, “must be that the choice is derived from the data” (ie. Wilkinson was cherry-picking).

When Judge recalculated the data based on the more usual measure of income per head, the association between life expectancy and inequality disappeared. Judge concluded:

In retrospect, it seems extraordinary that a predominantly monocausal explanation of international variations in life expectancy should ever have been regarded as plausible. It is much more likely that they are the product of many influences, which probably interact over long periods of time.

This was only common sense. Further studies conducted in Denmark and Japan failed to support Wilkinson’s hypothesis and although some studies showed an association between income inequality and life expectancy in the USA, other evidence showed that this was more likely to be due to education, underinvestment and other confounding factors. In 2002, a large study of wealthy European countries showed no association between inequality and life expectancy.

In The Spirit Level, Wilkinson and Pickett cite a 1996 editorial from the BMJ which discussed the “big idea” that “the more equally wealth is distributed the better the health of that society.” At that time, the BMJ was broadly supportive of the theory but research into it was still in its infancy. Wilkinson and Pickett do not mention the editorial that appeared in the same journal six years later, which concluded: 

Now that good data on income inequality have become available for 16 western industrialised countries, the association between income inequality and life expectancy has disappeared.

By the time The Spirit Level was published, the slender circumstantial evidence upon which Wilkinson had made his case had been obliterated with the passing of time. Rather than change the theory to fit the facts, he and Pickett ignored the facts and persisted with the theory. In practice, that meant using old data when they were quite aware that more recent data was available.

God and the movies

Maybe it's because I'm a bit of a statistics nerd, but once I start comparing countries, I find it difficult to stop. The fact is that associations are everywhere and the possibilities opened up by ecological epidemiology are endless. For example, here are two statistically significant associations showing belief in God and cinema attendance (against inequality).

Based on the World Values Survey. 'How important is God in your life?' Percentage answering 'very important'.

Based on Nationmaster data.

I show these graphs because they happen to show a strong correlations—stronger than most of The Spirit Level graphs. They are, as Wilkinson and Pickett might say, too strong to be the result of chance. But if they are not due to chance, how do we explain why people in less equal countries are more likely to believe in God and more likely to go the cinema? Does egalitarianism cause atheism? Does religion cause inequality? Or does going to the cinema make people believe in God? 

The possibilities are limitless. But even if you exclude chance as a possibility (and that would be very hasty), explaining them in terms of income inequality requires a vivid imagination and a near-obsession with wealth redistribution. 

As ever, our willingness to accept statistical associations depends on our susceptibility to the underlying message. Perhaps a socialist atheist would find these associations compelling. Or maybe a Christian film buff could use them as an argument in favour of capitalism. The rest of us might shrug our shoulders and say 'so what?'

 

US states

[This is a discussion of the issues mentioned on page 15 of The Spirit Level Delusion]

The Spirit Level Delusion focuses on differences between nations. It does not look at differences between US states. There are several reasons for this. 

Firstly, since—as I show—inequality does not influence outcomes on an international level, it is highly improbable that it would influence outcomes between regions within a country. Although the USA is a diverse place, there are greater cultural differences between, say, Israel and Japan than there are between Texas and California. Perhaps this explains why, even by the standards of The Spirit Level, the correlations shown for US states are weak. It is a stretch to say that the graphs they show on, for example, pages 59 and 141 are indicative of a genuine causal relationship.

Secondly, a discussion of US states would make the book twice as long and, for the reasons above, this would be an unnecessary strain on both reader and author.

Thirdly, and most importantly, the health and social problems highlighted in the USA are more plausibly associated with absolute income than with inequality. Wilkinson and Pickett deny this, but their own evidence contradicts them. On page 22 of The Spirit Level, they show two graphs. The first shows their index of health and social problems against inequality, the second shows the same index against 'national income' (this must be a mistake—they surely mean 'state income'). These data are shown below.

Wilkinson and Pickett state that the second of these graphs shows no association between income and health and social problems, but even the casual reader can see that this is not true. Clearly, the worst afflicted states are the poorest. Wilkinson and Pickett divert the reader's attention from this fact by not showing the linear regression line. Had they done so, the graph would have looked like this...

There is a reasonably strong correlation here. The poorest states do worse and none of the richer states do badly. It is, however, worth asking why the correlation is not even stronger. The answer lies in the dozen states in the bottom left corner. These states do not show a close fit, but that won't concern the people who live there. All these states are doing better than would be expected from their income. This is no great surprise. These are sparsely populated, predominantly rural, ethnically homogenous Northern states like North Dakota, Wyoming and Montana. No one pretends that wealth is the only factor that makes a state or country successful, but as this graph shows, it certainly helps. Look at the top right-hand corner. Those are the states that have higher state incomes but perform badly. Don't see any? Exactly. 

Connecticut, Massachusetts and New York hold the key to understanding this data. They are outliers in Wilkinson and Pickett's inequality graph (above) and consistently fail to follow the pattern in The Spirit Level's other graphs. How three very unequal states do so well is an important question which Wilkinson and Pickett never address, but the answer is simple. They do well because of their wealth—inequality is irrelevant. Massachusetts performs so well because it is the third richest state. Connecticut and New York are outliers on the inequality graph but are a perfect fit on the income graph. 

Mississippi and Louisiana perform very poorly under most criteria. They have the lowest life expectancies and the highest rate of infant mortality of any US states. But they are also the poorest and third poorest states respectively. This, more than inequality, must be considered the real driver of outcomes in these states. 

We cannot rule out some indirect effect from inequality, but any effect is far from proven. The apparent correlation between inequality and outcomes is not, in itself, reason enough to cry causation. Other factors, such as the proportion of African-Americans in the population (shown below), show a stronger correlation but here, too, it would be foolish to insist on a direct causal relationship. 

Many other such graphs could be created showing strong correlations—some valid, some spurious—and that is only to be expected. Outcomes in societies are dependent on countless cultural, political, economic and demographic factors that interweave over many years. This is an obvious point to make, but in the light of The Spirit Level's monocausal 'theory of everything', it is one that needs to be reasserted.

Graphs and sources

All the statistics in The Spirit Level Delusion are there to be checked. Here are the sources...

Life expectancy

See Chapter 1 of The Spirit Level Delusion and case study.

Source: UN Human Development Report (2006)

Quality of life

See Chapter 3 of The Spirit Level Delusion

Source: The Economist Intelligence Unit’s quality-of-life index, 2005

Homicide (without USA)

See Chapter 4 of The Spirit Level Delusion

The 'link' between inequality and homicide rests on the USA's unusually high murder rate. When this obvious outlier is excluded, the association disappears. A correlation that depends on one outlier is, of course, no correlation at all. 

All this data comes from the 1990s. Portugal's murder rate has since fallen to around the same level as Sweden, further undermining the inequality hypothesis.

Source: United Nations Surveys on Crime Trends and the Operations of Criminal Justice Systems (CTS) (1990-2000 average)

Involvement in the community

See Chapter 3 of The Spirit Level Delusion

Source: World Values Survey (go to 'Online data analysis') 

Happiness

See Chapter 3 of The Spirit Level Delusion

Source: World Values Survey (go to 'Online data analysis') 

Happiness against national income

See Chapter 3 of The Spirit Level Delusion

Contrary to Wilkinson and Pickett, there is ample evidence that economic growth benefits the population even at a very high level of development. The happiness survey is one example of this.

Source: World Values Survey (go to 'Online data analysis') 

National income figures: World Bank 2008

Births to women aged 15-19 years

See Chapter 3 of The Spirit Level Delusion

There is no association with inequality for the majority of countries. Rates tend to be somewhat higher in English-speaking countries, but the rate in countries like Hungary, Hong Kong and Singapore suggest that this is not due to inequality. The slightly higher rate in Portugal is more likely to be due to the higher incidence of teen marriages and abortion being illegal when this data was collected.

Source: UNICEF and United Nations

Divorce

See Chapter 3 of The Spirit Level Delusion

Source: Americans for Divorce Reform

Prisoners per 100,000

See Chapter 4 of The Spirit Level Delusion

Source: United Nations Surveys on Crime Trends and the Operations of Criminal Justice Systems (CTS) (Ninth edition, or earlier when unavailable)

Recorded crime per 100,000

See Chapter 4 of The Spirit Level Delusion

There is an inverse relationship between inequality and the crime rate. There is also—but not always—an inverse relationship between the prison rate and the crime rate.

Source: United Nations Surveys on Crime Trends and the Operations of Criminal Justice Systems (CTS) (Ninth edition, or earlier when unavailable)

Education

See Chapter 9 of The Spirit Level Delusion

The countries of Southern Europe tend to perform least well but the performance of places like Hong Kong and Australia—as well as the lack of any statistically significant association—strongly suggest that inequality is not the cause of this.

Source: The Programme for International Student Assessment (PISA). Combined maths, literacy and science scores (2006)

Government spending on foreign aid

See Chapter 6 of The Spirit Level Delusion

Source: OECD (2008)

Per capita donations to charity

See Chapter 6 of The Spirit Level Delusion.

The trend towards more equal countries giving more in state aid is counter-balanced by the tendency for less equal countries to give more in individual, voluntary donations.

Source: Salaman (1999). See also Charities Aid Foundation (2006)

Sources for other graphs shown in The Spirit Level Delusion:

Inequality: UN Human Development Reports. Same methodology used as in The Spirit Level.

Smoking rates: Nationmaster

Unemployment: UN Human Development Reports.

Alcohol Consumption: World Health Organisation (2004)

Infant mortality: United Nations World Population Prospects (2008) 

Obesity: International Obesity Taskforce 

Recycling: Planet Ark (2004)