The Spirit Level has been debunked. More or less.

The BBC Radio 4 show More or Less looks at the facts behind well publicised statistics. The show is presented by the author of The Undercover Economist, Tim Harford and, as a regular listener, I was intrigued to hear that the first show of the new series promised to 'decode The Spirit Level debate'. For a show dedicated to debunking junk statistics it was obvious subject matter, but I wondered how Harford could 'decode' such a voluminous topic over the airwaves in one show.

In the end, he didn't need to. From the outset, Harford admitted there were too many competing claims to fit into a magazine show, and instead interviewed The Spirit Level's co-author Kate Pickett, who did more damage to the reputation of The Spirit Level in the space of ten minutes than any number of supposed "idea wreckers".

The interview (listen here for the next couple of days, or subscribe to the podcast) is essential listening if you've been following the controversy, as Pickett struggles to answer some fair and simple questions. Strangely enough, Wilkinson and Pickett's Equality Trust website—which is normally so quick to let people know when The Spirit Level has been mentioned in the media—have yet to post a link to this interview. And since the audio file won't be available for much longer, I've transcribed some of the key moments for posterity...


Falling back on other people's research

Wilkinson and Pickett's first line of defence is to claim that there are 100s of peer-reviewed studies which support their conclusion. As I have said before, this is just not true. Most of the studies they reference in The Spirit Level do not even mention income inequality.

In the More or Less interview, Kate Pickett once again claimed there was a "vast body of research" behind The Spirit Level. Tim Harford picked her up on it...

KP: We wrote a book that's intended to be a synthesis of a very vast body of research. Not only our own, but those of other people... There is a consistent and robust and large body of evidence showing the same relationship.

TH: That's an interesting point that you make. Often, in response to critics, you have referred not to your own book, not to your own data but to other published research. I'd really like to focus on the research that's presented in your book. It's very easy to say there are 50 papers, there are 200 papers, that support our research but we don't really know how you've selected those papers.

KP: We actually have completed a systematic review of all of the studies of income inequality and health, and we reference that in our book. We do examine things systematically and certainly—when we are doing our own research, publishing in peer-reviewed journals—we have to be aware of all the literature in the field. But that doesn't mean that every paper in the field has good methods, comes to the right conclusion, studies the right thing.

TH: I absolutely agree. One of the papers that you refer to in support of your argument on women's empowerment and women's status which was published in 1999 by Kawachi and some other authors, you claim supports your findings on women's status and income inequality. I've looked at their abstract. It doesn't seem to attack that question at all. It's simply on another subject—a somewhat related subject but not on the subject of income inequality.

KP: They've definitely published and we may have inadvertently put the wrong reference into that document [laughing nervously]. But Kawachi and Kennedy have certainly published finding a relationship between income inequality and women's status. The paper is Women's Status and the Health of Women and Men: a view from the States and it was published in Social Science and Medicine in 1999.

TH: That's the one I'm looking at.

Note: On page 58 of The Spirit Level, it states: "Researchers at Harvard University showed that women's status was linked to state-level income inequality. (36)"

Reference 36 is the Kawachi study ('Women's status and the health of women and men: a view from the States', 1999). As its title suggests, this study compared women's status with health, not with inequality. Indeed, the authors found a correlation between women's status and health even after controlling for income inequality.

Failure to look at other variables

The Spirit Level relies on the conceit that countries are fundamentally the same, with income inequality being the main variable that distinguishes them. This allows Wilkinson and Pickett to disregard other variables such as income, culture, history, demography, ethnicity, geography, law, politics and climate. Ignoring other variables and confounding factors would be a flaw in any study—as Harford points out, it breaks a basic rule of epidemiology—but when entire countries are being studied, this flaw becomes overwhelming. Pickett's response is revealing: she and Wilkinson do not "believe" that factors other than income inequality have an effect on a country's performance, so they don't bother looking at them.

TH: All of your studies are what are called bivariate analysis. In other words, they're all income inequality plotted against some other variable. Now, my understanding of best practice in social sciences is that you would always control for other variables. You would include 2, 3, 4, 5, 6 other variables and...

KP: Well, you wouldn't do that arbitrarily. You would do that if you believed those variables were potential alternative explanations of the relationship you're looking at.

TH: So, if I understand your statement correctly, you didn't include any multiple variable analysis because you just think that actually none of these variables are of interest—none of them are potential alternative explanations and you can just do the simple income inequality versus x analyses?

KP: That's right, but of course, again, other researchers have conducted studies that do control for more, where, as well as examining the effect of income inequality at the level of the whole society, people include individual's own levels of income or levels of education in those analyses and, again, those bear out our findings in relation to health.

TH: We come again to...you're basically rowing back from your analysis and saying...

KP: No. Indeed I'm not...

TH: "Don't look at our analysis, look at these other people because they support us."

KP: We believe that to control for individual income is actually over-controlling, so we would not consider that best practice.

Academic criticism

Although well received by some journalists and politicians, The Spirit Level has received a much cooler reaction from academics. One of the few serious academics to have reviewed the book was John Kay, Professor of Economics at London Business School and former Director of Institute of Fiscal Studies. Pickett's response to Kay's review speaks volumes...

TH: When John Kay reviewed your book in the Financial Times —and I believe John Kay would be broadly sympathetic to your idea that egalitarianism is important—he wrote: "The evidence presented in the book is mostly a series of scatter diagrams with a regression line drawn through them. No data is provided on the estimated equations, or on relevant statistical tests. If you remove the bold lines from the diagram, the pattern of points mostly looks random, and the data dominated by a few outliers." Do you think that's fair?

KP: No, I don't think it's fair. [Testily] He didn't read the book thoroughly, obviously.

Outlandish claims 

In the last chapter of The Spirit Level, Wilkinson and Pickett make some extraordinary claims about what could happen if Britain reduced income inequality to Scandinavian levels. These include: teen births falling to a third of current rates, mental illness being halved, life expectancy rising by a year and the murder rate falling by three-quarters. Harford asks her about the last of these predictions.

TH: Clearly your book is a systematical analysis and partly also a political book. You have a political case to make—there's nothing wrong with that. You have public policy actions that you would like to see taken. But do you think you may have overstated some of those? Let me give you an example. On page 268 of you book—towards the conclusion—you say that if Britain became as equal as Japan, Norway, Sweden and Finland, homicide rates could fall by 75%. But as I'm sure you've had pointed out to you by now, the UK's homicide rate is already below the average of those four countries.

KP: It's not actually. It's been pointed out that it's below Sweden. It's not below the average of those countries. Those claims [ie. in The Spirit Level], they're based on regression models and of course they're only as good as they're model they're based on.

TH: [Incredulously] But.. sorry... but you've made that claim!

KP: Yes, yes, we do...

TH: And you stand by it?

KP: Yes. That Britain would become a much healthier and more socially better functioning place if it were more equal.

TH: You said that if Britain became as equal as these four countries, homicide rates could fall by 75%. Do you not feel that's really overstating the case, or do you stand by that?

KP: That's based on the model. I mean, I think we could try it and see.

[end of interview]

TH: Kate Pickett, co-author of The Spirit Level. We did go to her Equality Trust website, by the way, and downloaded the data on homicide rates in the UK and in the relevant four countries and it does seem that I was right to say that the UK's homicide rate is already below the average of those four countries. You're listening to More or Less...

Today's report in The Guardian

Today's report in The Guardian will no doubt draw the usual hate and bile from people who have no intention of reading my book. Still, a few quick points...

Nobody asked me or paid me to write this book. I never set out write a critique of another book. While I was researching a completely different topic, I bought and read The Spirit Level because, as I said to The Guardian, it was "influential and informing debate." Those are the kinds of books I like to read, whether from left or right. When I started fact-checking The Spirit Level I realised that it was too big a subject to squeeze into an article or blog post and The Spirit Level Delusion was born.

When Dr Patrick Basham kindly offered to write the preface for the book, I published it in association with the Democracy Institute, of which he is the director. Had I known this would leave the book open to accusations of being written by a "wrecker" from a "rightwing thinktank" I wouldn't have bothered. You live and learn.

I knew when I wrote it that the dogmatic right wouldn't be interested because they wouldn't have read The Spirit Level. I knew the dogmatic left wouldn't be interested because they'd put their fingers in their ears if anyone raised difficult questions about such a politically useful text. But I also knew that there would be some people in between who had enquiring minds and a genuine interest in the issues. Perhaps I overestimated how many fell into that camp.

The Guardian quoted a few words from a twenty minute interview. No complaints, that's the way it goes...

He [Snowdon] does not believe that The Spirit Level's claim that the psychological effects on society of income inequality are so great to cause widespread social ills. "I don't think people outside the intelligensia worry about inequality," Snowdon said. "The working class don't worry about how much Wayne Rooney is earning."

It's a crude example, but it serves to illustrate one of the fundamental problems with The Spirit Level. It cannot be stressed often enough that Wilkinson and Pickett's hypothesis rests on the psychological (or 'psychosocial') effects of living in a less equal society, not the material effects of poverty.

When people say that they find The Spirit Level's conclusions to be 'intuitively' true, or that they appeal to 'common sense', I wonder whether they fully appreciate that Wilkinson and Pickett are not blaming poverty, low income or low living standards per se. They are talking about something much less tangible—a sense, a feeling, a response—to other people's wealth. As someone who happens to be in the bottom 20% of earners myself, I don't personally feel traumatised by the existence of the super-rich. Perhaps that's just me, but there is also very little empirical evidence that the psychological response to inequality has a significant effect on people's day-to-day lives.

Wilkinson and Pickett would disagree, but the (left-wing) economist JK Galbraith understood this back in 1958 when he wrote The Affluent Society:

Envy almost certainly operates efficiently only as regards near neighbours. It’s not directed towards the distant rich.

In a later preface to The Affluent Society, Galbraith returned to the issue of inequality, making it clear that so long as people's own living standards were improving, they are not troubled by the thought of other people becoming still richer:

When, as suggested in this book, men and women are employed and at continuously improving wages or salaries, they are not greatly concerned that others, with whatever justification or absence of justification, have more, even greatly more.

More recently, in Status Syndrome, the (left-wing) epidemiologist, Michael Marmot discussed the stubborn refusal of ordinary Americans to become less happy even as their country became less equal. He made a telling comment about who is really 'stressed' by income inequality:

Changes in income inequality did not affect happiness levels of the poor. The subgroup of the population whose happiness declined when income inequality increased, were richer people who described themselves as on the left politically.

I discuss this issue in more detail in the later chapters of the book.

No doubt there is resentment at some of the grotesque disparities of wealth that exist (and have always existed), but that resentment would have to be truly monumental for it to be the main driver of an entire country's performance across so many criteria. Very few variables—let alone psychological variables—show up in aggregate data from whole nations. The psychosocial effect of income inequality is not one of them, and Wilkinson and Pickett have to perform all sorts of twists and turns to make their case to the contrary. At best, The Spirit Level gives a cock-eyed view of the way the world is.

The case for greater income equality remains an ethical, moral and political issue. It cannot be 'proved' by social science.


My response to Wilkinson and Pickett's answers to my 20 Questions is here .

Some of the graphs from The Spirit Level Delusion are here.

Links to other sites discussing the debate over The Spirit Level can be found on the right-hand side of the page.

Reply to Prospect magazine article

This is my reply to an article in Prospect by Richard Wilkinson and Kate Pickett. That article can be read here. The article that inspired it is here.


I’m pleased to be able to agree with Wilkinson and Pickett (W & P) on one point. I’m not a public health researcher and was surprised to be described as such. That, sadly, is where agreement ends.

Wilkinson and Pickett once again imply that they are merely the messengers of a scientific consensus and that there are 100s of peer-reviewed studies saying the same thing. Can we please put this one to bed? Even if quantity was a substitute for quality, the argument does not apply here. There is a large body of conflicting research about health and inequality and a smaller body of research studying violence and inequality. Both are hotly debated, not least because it is very difficult to isolate the effects of income inequality from the effects of low income.

Beyond this, Wilkinson and Pickett are out on their own, making claims that have virtually no support in the scientific literature. In contrast to what he says in the third paragraph of this rejoinder, Wilkinson recently told the magazine International Socialism:

"There are about 200 papers on health and inequality in lots of different settings, probably 40 or 50 looking at violence in relation to inequality, and very few looking at any of the other things in relation to inequality. In a way, the new work in the book is all these other variables—teenage births, mental illness, prison populations and so on—and the major contribution is bringing all of that into a picture that had previously been just health and violence."

W & P confuse making assumptions based on other people’s research with having those people actually agree with them. For example, they cite studies that quite reasonably associate overeating with stress, but it does not follow that obesity rates vary internationally because the population is stressed about inequality. At best, this is speculation.

W & P continue to cite perfectly sound studies showing there to be social gradients to health and social problems as evidence that inequality affects a nation’s overall performance. It does not. These are completely different issues.

Which leaves us with W & P’s own evidence, which relies on comparing whole countries, a notoriously unreliable method which allows unlimited scope for misinterpretation. The criticisms recently made of this evidence by myself and others closely echo criticisms made in peer-reviewed journals when Wilkinson used similar methods in the past. They also echo criticisms made by the few serious academics who have reviewed The Spirit Level.

Anyone who believes that W & P “never pick and choose data points to suit our argument” should compare references 2 and 6 in The Spirit Level (p. 271) and ask themselves why one year’s data were used for one graph and another year’s data used for the next. Anyone who believes that they use “the same measures of inequality” should turn to page 224 and ask why a dramatically different measure of inequality was preferred when working hours were studied (clue: see how it looks when we use W & P’s more usual measure of inequality). Anyone believing that they have not “picked problems to suit our argument” might ask why they show how much overseas aid is given by a country’s government, but do not show how much is given privately (there is no correlation with inequality when the two are combined).

As for always using the same group of countries, one of The Spirit Level’s most serious flaws is the baffling assumption that “rich market societies” come in batches of 50. If there is to be a cut-off point beyond which economic growth has “largely finished its work”, it should be based on something more than a round number. Without a convincing justification for why places like the Czech Republic and South Korea – let alone Hong Kong – cannot be considered rich market societies, we must ask the next question: why do these societies conspicuously fail to fit Wilkinson and Pickett’s theory? The United Nations classes these countries as being of “very high human development”, why doesn’t The Spirit Level?

I hope that readers will take the time to look at these issues themselves, but, if not, they should at least take a deep breath and ask themselves which is more plausible – a theory that seeks to explain the workings of whole societies by reference to a single factor, or one that says that a country’s performance is the result of countless historical, geographical, political, legal, demographic and economic factors, of which the public’s response to income inequality may or may not be one.