The illusion of consensus

A key plank in Wilkinson and Pickett’s defence of The Spirit Level is the notion that they are merely informing the general public about issues that have long since been agreed upon by the academic community. Since most people will never read any of the studies in the field, this has been largely successful as a public relations exercise, but it is a gross distortion.

It also represents something of a U-turn for the two social epidemiologists. Wilkinson and Pickett’s sudden insistence that they are reflecting the scientific consensus is at odds with the way they promoted their book when it was first released. In an interview with the couple in March 2009, the Guardian journalist reported that:

For a while, Wilkinson and Pickett wondered if the correlations were too good to be true. The links were so strong, they almost couldn't believe no one had spotted them before.

This could just about be excused as shoddy journalism were it not for Wilkinson and Pickett’s eagerness to take the credit for what they explicitly described as their “discoveries” in The Spirit Level itself. The book’s preface leaves the reader in little doubt that what they have discovered is genuinely new and exciting, hence the comparisons with Joseph Lister and Louis Pasteur. “The reason why the picture we present has not been put together until now is probably that much of the data has only become available in recent years,” they write, adding that “it could only have been a matter of time before someone came up with findings like ours.”

The truth of the matter, as discussed in Chapter 1 of The Spirit Level Delusion, is that there has been a large amount of research into the specific area of health and inequality spanning three decades. Richard Wilkinson has been a key figure in this field, but his views do not represent the consensus. Not could they, since there is emphatically no consensus. The only honest way to describe the state of the literature on health and inequality is to say it is mixed and conflicting. Researchers are broadly divided into three groups. There are those, like Wilkinson, who believe that there is a solid correlation between inequality and health outcomes and that this represents a causal link. There are those who believe there is a statistical correlation but that it is not causal, and there are those who believe there is no link at all.

Only the first of these positions is reflected in The Spirit Level, and the reader is given the false impression that academics have firmly established that inequality leads to poor health. Wilkinson and Pickett accuse their critics of not being familiar with the “extensive research literature”, but it is precisely because we are familiar with it that we know how grievously the pair misrepresent it in The Spirit Level. In the new postscript to the book (published November 2010), Wilkinson and Pickett say that “there are around 200 papers in peer-reviewed academic journals testing the relationship between income inequality and health”.

‘Testing’ is the key word here. There is no hint of how many of these studies have not found a relationship, nor of how many found a statistical relationship but concluded that it was not a causal. Their source for the ‘200 studies’ claim is, as so often in the book, one of their own papers. This article, from 2006, assessed 169 results from 155 studies on inequality and health (plus some other studies related to violence). By Wilkinson and Picket’s own reckoning, 88 of these were supportive of their theory (including 6 of their own studies) while 81 were either unsupportive or inconclusive.

Wilkinson and Pickett stress that many peer-reviewed articles have offered at least partial support to the relative income hypothesis. This is true—at least in the area of health—just as it is true that there are many peer-reviewed articles that beg to differ. Hence the long-running academic debate about inequality which The Spirit Level has done much to popularise but little to resolve. This debate has already been discussed in Chapter 1, but it might be useful to quote from some other researchers in the field:

All along, however, critical questions were being asked about the quality and interpretation of the data. In an early exchange, serious criticisms of the selection of countries, the quality of the data, and the lack of control for confounding in the BMJ paper of 1992 were only half countered. Although many aspects of this debate are still unresolved, it has recently become clear that the findings of that paper were an artifact of the selection of countries.

—British Medical Journal editorial, 2002

This paper extends previous studies by examining long time series for 12 of the world’s richest countries rather than one or two. Our findings are consistent with those of Deaton and Paxson (2001) and Lynch et al. (2004b), not with those of Wilkinson (1989, 1996) or Sen (1999). In our preferred specifications we find only small and statistically insignificant relationships between income inequality and mortality. This holds true regardless of whether we measure mortality using life expectancy at birth, infant mortality, homicide, or suicide.

—Leigh & Jencks, 2007

The study found limited evidence of an association between income inequality and worse self rated health in Britain, which was greatest among those with the lowest individual income levels. As regions with the highest income inequality were also the most urban, these findings may be attributable to characteristics of cities rather than income inequality. The variation in this association with the choice of income inequality measure also highlights the difficulty of studying income distributions using summary measures of income inequality.

—Weich et al., 2002

Estimates of the effect of income on health (the absolute income hypothesis) are likely to be biased. Tests of the relative income hypothesis are contaminated by the non-linearity of the individual health income relationship any association between income distribution and population health could be entirely due to it, rather than to any direct erect of relative income on individual health.... However, whilst Rodgers (1979) found that income distribution had a significant negative association with life expectancy in almost all of his regression, we have found that the association is sometimes positive and sometimes negative and is never statistically significant.... The findings should however be a further warning against using aggregate level studies as evidence for the relative deprivation hypothesis.

—Gravelle, 2000

Income inequality was not associated with health status... Household income, but not income inequality, appears to explain some of the differences in health status among Canadians.

McLeod et al., 2003

Significant differences in income inequality across regions and considerable changes in health are found across years, however, the panel data estimating regressions find no significant association between any of the measures of income inequality and self-reported health. Therefore, it would appear that the relative income hypothesis does not exist over time and does not exist within Britain.

Lindley & Lorgelly, 2005

Across Canadian health regions, health status in populations was a function of absolute income but not relative income.

—Vafaei et al., 2010

It can be firmly concluded, however, that there is insufficient evidence supporting Wilkinson’s hypothesis once individual’s income and its differential impact are taken into account... There are substantial international variations in self-reported health, but they are not linked to the degree of income inequality... Wilkinson’s argument regarding contextual influences was based on a statistical artifact.

—Jen et al., 2009

Those with a healthy scepticism will have noticed that I have only quoted studies that support one side of the debate. It’s a slippery and misleading trick and it is exactly what Wilkinson and Pickett do throughout The Spirit Level. The difference is that I made it clear from the outset of this book that there are many conflicting studies. Readers of The Spirit Level would be hard-pressed to guess that there was any debate at all.

In their new postscript and in response to an article I co-wrote for the Wall Street Journal, Wilkinson and Pickett cite a 2009 review of self-reported health studies in the British Medical Journal which, they say, "leave[s] little room for doubt as to the veracity of these relationships [and] shows unequivocally that inequality is related to significantly higher mortality rates." With so many studies to chose from, it is reasonable to expect Wilkinson and Pickett to cite one which strongly supports their position. But while the BMJ study is more supportive than most, it can hardly be called unequivocal. It begins by noting that:

Empirical studies have attempted to link income inequality with poor health, but recent systematic reviews have failed to reach a consensus because of mixed findings.

And concludes:

The results suggest a modest adverse effect of income inequality on health, although the population impact might be larger if the association is truly causal... The findings need to be interpreted with caution given the heterogeneity between studies.

It says much how about how weak the alleged ‘consensus’ is that the study Wilkinson and Pickett use as killer proof that inequality causes poor health did not find a strong relationship and acknowledged that the “modest” association was weak enough to imply a lack of causality. If this is “unequivocal” evidence, what is the rest like?

Other researchers who have reviewed the evidence have not been so generous. For example:

Only individual-level studies have the potential to discriminate between most of the advanced hypotheses. The relevant individual-level studies to date, all on U.S. population data, provide strong support for the “absolute-income hypothesis,” no support for the “relative-income hypothesis,” and little or no support for the “income-inequality hypothesis.”

—Wagstaff & Doorslaer ('Inequality and Health: What does the literature tell us?')

The undeniable absence of a strong or consistent relationship between inequality and health stands in stark contrast to previous claims.... Contrary to the claims of previous researchers, there is no strong empirical support for the contention that inequality is a determinant of population health, let alone one of the most important determinants.

—Mellor and Milyo ('Reexamining the Evidence of an Ecological Association between Income Inequality and Health')

This article reviews 98 aggregate and multilevel studies examining the associations between income inequality and health. Overall, there seems to be little support for the idea that income inequality is a major, generalizable determinant of population health differences within or between rich countries.

—Lynch, ('Is income inequality a determinant of population health?')

Much of the literature, both theoretical and empirical, needs to be treated skeptically, if only because of the low quality of much of the data on income inequality. Although there are many remaining puzzles, I conclude that there is no direct link from income inequality to mortality; individuals are no more likely to die or to report that they are in poor health if they live in places with a more unequal distribution of income.

—Deaton ('Health, Inequality, and Economic Development')

The last quoted paragraph comes from a review of the literature conducted by Prof. Angus Deaton of Princeton University, one of the world’s most respected economists, whose summary of the evidence has twice as many citations in the scientific literature as Wilkinson and Pickett’s 2006 paper. Despite this, the postscript to The Spirit Level finds Wilkinson and Pickett stating that “it is now extremely difficult to argue credibly that these relationships don’t exist. Indeed, those who do so are almost always those who are making political attacks rather than any kind of academic criticism.” This statement goes beyond the merely misleading and enters the realms of flagrant dishonesty. In 2009, The Oxford Handbook of Economic Inequality evaluated the evidence for the inequality-health hypothesis and concluded:

The preponderance of evidence suggests that the relationship between income inequality and health is either non-existent or too fragile to show up in a robustly estimated panel specification. The best cross-national studies now uniformly fail to find a statistically reliable relationship between economic inequality and longevity.

Having to resort to the appeal to authority is regrettable, but since Wilkinson and Pickett are so eager to bill themselves as “epidemiologists with decades of experience in analysing the social determinants of ill health”, it behooves me to be said that each chapter of The Oxford Handbook of Economic Inequality is written by a team of distinguished professors who are regarded as international experts in their field. The implication that the work of these eminent scholars is “ill-founded and politically motivated criticism” is risible. Unlike Wilkinson and Pickett, none of these academics have formed any political pressure groups and do not have a long history of demanding radical wealth redistribution.

As Sanandaji et al., have noted, the idea that Wilkinson and Pickett took their message directly to the public only after winning the academic debate is one of The Spirit Level’s most enduring myths:

The general public—the target audience for The Spirit Level—cannot be expected to be aware of the state of research in the field. Wilkinson and Pickett exploit the trust of their readers and give them the impression that their claims represent consensus science, when the opposite is closer to the truth.

Wilkinson and Pickett totally misrepresent the literature on inequality and health in The Spirit Level. They build the illusion of consensus around the one criterion that has generated substantial academic study (health) without ever acknowledging that the inequality-health hypothesis remains highly controversial and that Wilkinson's attempts to 'prove' it have attracted much criticism in the peer-reviewed literature spanning two decades.

Having given a distorted and one-sided account of the research into health and inequality, they then lead the reader to believe that there is also a "vast literature" supporting their claims about other criteria. In fact, the amount of published research into these other criteria range from scant (eg. infant mortality, obesity, teen births) to none at all (eg. foreign aid, recycling, innovation). Wilkinson and Pickett's misrepresentation of the work of other academics will be the subject of the next post.