Ignoring crucial facts

Any theory which explains the working of entire nations by looking at just one variable should strike us as being inherently questionable. We know that societies are moulded by a huge range of complex factors which come together over long periods of time. Some are accidents of circumstance, some are flukes of geography, history, climate or demography. Others are come about through the force of politics, religion or industry. However they come about, it is far from controversial to say that societies across the world are different for many reasons.

The Spirit Level relies on the premise that countries are fundamentally the same, with income inequality being the main variable that distinguishes them. Wilkinson and Pickett effectively disregard other variables such as absolute income, culture, history, ethnicity, geography, law, politics and climate. Throughout The Spirit Level, it is taken for granted that such factors have little or no bearing on their findings and so there is no attempt to adjust the figures for confounding factors, or even discuss them.

In the new postscript to the book, Wilkinson and Pickett group all these other variables together and dismiss them as “cultural differences” which, they say, have a negligible effect on their findings. To illustrate this, they say that Portugal and Spain perform very differently despite being culturally similar, while Japan and Sweden perform similarly despite being culturally different. This is not true. In most of the graphs, Portugal is actually closer to Spain than Japan is to Sweden.

More telling wold be a comparison between Japan (the most equal country) with Hong Kong and Singapore (the least equal countries). Despite the huge disparities in income inequality, these three societies perform much the same across nearly all criteria (imprisonment being the main exception). The obvious explanation is that these Asian societies are culturally similar.

Ignoring other variables and confounding factors would be a flaw in any study but when entire countries are under examination, this flaw becomes overwhelming. Tim Harford asked Pickett about their failure to consider other variables on More or Less. Her response was revealing. She and Wilkinson did not “believe” that factors other than inequality have an effect on a country's performance, so they didn’t go to the trouble of studying them.

TH: All of your studies are what are called bivariate analysis. In other words, they're all income inequality plotted against some other variable. Now, my understanding of best practice in social sciences is that you would always control for other variables. You would include 2, 3, 4, 5, 6 other variables and...

KP: Well, you wouldn't do that arbitrarily. You would do that if you believed those variables were potential alternative explanations of the relationship you're looking at.

TH: So, if I understand your statement correctly, you didn't include any multiple variable analyses because you just think that actually none of these variables are of interest—none of them are potential alternative explanations and you can just do the simple income inequality versus x analyses?

KP: That's right, but of course, again, other researchers have conducted studies that do control for more, where, as well as examining the effect of income inequality at the level of the whole society, people include individual's own levels of income or levels of education in those analyses and, again, those bear out our findings in relation to health.

TH: We come again to...you're basically rowing back from your analysis and saying...

KP: No. Indeed I'm not...

TH: "Don't look at our analysis, look at these other people because they support us."

KP: We believe that to control for individual income is actually over-controlling, so we would not consider that best practice.

Wilkinson and Pickett may not believe that individual income explains any of the differences between the countries they study, but while this is taken for granted in The Spirit Level, it is not unreasonable to take the view that social outcomes in Portugal, for example, would improve if its national income was the same as Norway’s (which would require a threefold increase in wealth).

Pickett is, however, correct in saying that other researchers have controlled for other variables. Shibuya et al., for example, controlled for income in their study of inequality in Japan and concluded:

After adjustment, individual income was more strongly associated with self-rated health than income inequality.

Fiscella and Franks controlled for income in their study of inequality in the USA and found:

In this nationally representative American sample, family income, but not community income inequality, independently predicts mortality. Previously reported ecological associations between income inequality and mortality may reflect confounding between individual family income and mortality.

Absolute income is a crucial confounding factor in studies of income inequality. Much of the debate about inequality and health revolves around the question of whether we can truly disentangle the effects of inequality from the effects of low income. Wilkinson and Pickett completely overlook this issue, and they never remark on the important observation that the poorest countries in their list (Portugal, Greece and New Zealand) all happen to be ‘less equal’. Nor do they comment on the fact that the perennially underachieving US states of Alabama, Louisiana and Mississippi also happen to be amongst the very poorest.

From the outset, income is assumed to have no role to play in The Spirit Level. Having announced that economic growth has “largely finished its work”, Wilkinson and Pickett simply assume that further wealth would not benefit the citizens of the countries they study (another glaring ecological fallacy, incidentally). It is assumed that absolute income has no effect because—as they show on page 12—life expectancy is no longer correlated with national income. But they do not test every criteria against income. If they did, they would find that several key outcomes are much more closely correlated with income than with inequality. This is true even of their cherished survey about trust, as the graphs below show. The first shows trust against national income; the second shows trust against income inequality.

Having breezily dismissed income as a third variable, Wilkinson and Pickett turn a blind eye to all other explanations for a country’s performance. Indeed, the only examples of them mentioning real-world differences occur when the ‘more equal’ countries fail to live up to their billing of ‘almost always’ doing better. For example, Wilkinson and Pickett are eager to explain Finland’s high homicide rate by pointing to its high level of gun-ownership while blaming the USA’s high homicide rate squarely on inequality. When Japan’s foreign aid contributions turn out to be “lower than expected”, they attribute it to the country’s “withdrawal from the international stage following the Second World War”. Britain’s “higher than expected” foreign aid spending, on the other hand, is explained by its “historical, colonial ties to many developing countries.” All of this may be true but Wilkinson and Pickett only seem aware of cultural and historical differences when it suits their argument.

In reality, of course, they know perfectly well that other variables have been shown to explain differences between countries far more convincingly than inequality. In their 2006 review of the literature, they identified 21 studies which “started off with supportive findings but then lost them as a result of the various control variables.” Income is one of those variables, but other recognised confounders include spending on health care, which has been found to explain the correlation between inequality and infant mortality:

The association of higher income inequality and higher infant mortality disappears when we control for health care expenditure. Our results indicate that the correlation between infant mortality and income inequality arises as income inequality is high in countries where public investment in health care is low.

And:

Although income inequality was positively associated with low birth weight and infant mortality, the association with infant mortality disappeared with the addition of sociodemographic covariates.

Levels of education have also been shown to explain correlations with inequality:

Multiple regression analysis of the 50 US states and District of Columbia for 1989-90 indicates that the relation between income inequality and age adjusted mortality is due to differences in high school educational attainment: education absorbs the income inequality effect and is a more powerful predictor of variation in mortality among US states.

Race is another important variable which is never adequately addressed in The Spirit Level. For example, one of the few studies looking at inequality and obesity acknowledged that:

Race is known to be significantly correlated with weight status, and is also associated with inequality... As race is a potential confounder of the relationship of interest, we stratify all analyses by race as well as sex.

The results of this study are worth repeating, since the they is ignored in The Spirit Level, in favour of Pickett’s own research:

We do not find a positive association between inequality and the likelihood of clinically relevant outcomes such as overweight and obesity. Indeed, the direction of association between inequality and weight status is generally negative among subgroups (though significant only for white women)... at least for non-Hispanic white women, living in a metropolitan area with greater income inequality is associated with lower BMI, lower odds for being overweight, and lower odds for being obese. [Emphasis in original]

Race has been shown time and again to be a major confounder in studies of inequality, to the extent that this one variable explains the entire correlation between inequality and poor health. This has been shown to be true in the USA:

In the results presented below, we show that, once we control for the fraction of the population that is black, there is no relationship in 1980 nor in 1990 between income inequality and mortality across either states or cities... That the estimated effects of income inequality are potentially confounded by the effects of race has been recognized since the first papers on the topic. Blacks have higher mortality rates than whites and, on average have lower incomes, so that in places with a substantial black population, both income inequality and mortality, tend to be higher.

In Canada:

We replicate the finding that, net of the racial//ethnic composition of the population, the effects of income inequality are not significant.

And in New Zealand:

There is no convincing evidence of an association of income inequality within New Zealand with adult mortality. Previous ecological analyses within New Zealand suggesting an association of income inequality with mortality were confounded by ethnicity at the individual level.

The well-established importance of race as a confounding factor provided Wilkinson and Pickett with the excuse to land their lowest blow yet. In his book Beware False Prophets, Peter Saunders demonstrates that health and social outcomes are more closely correlated with the ethnic make-up of US states than with their levels of income inequality. For this, Wilkinson and Pickett accused him of a “seriously racist slur”. It was, they said, “racist because it implies the problem is inherently the people themselves rather than their socioeconomic position”.

It implied nothing of the sort. If Wilkinson and Pickett think it is racist to say that there are a host of cultural and historical reasons why blacks tend to do worse than whites in the USA, then there are plenty of black community leaders and black politicians who are racist. No serious discussion of modern-day America can ignore the legacy of slavery and segregation, as well as the more subtle forms of ongoing discrimination which continue to hold African-Americans back. Black Americans have, on average, higher rates of obesity, higher homicide rates and lower life expectancy. It should, therefore, be no surprise that states with large black populations tend to do worse under these criteria.

There is no doubt that racial inequality contributes to income inequality. Wilkinson and Pickett argue instead that income inequality is, at heart, the cause of racial inequality. Aside from being counterintuitive, this cannot be so because the correlation between race and health and social problems is stronger than the correlation with income inequality.

A significant clue lies in the pages of The Spirit Level itself. Wilkinson and Pickett’s discussion of mental health is a mass of contradictions. Having warned of the dangers of comparing apples and oranges, they proceed to do just that by cobbling together results from different studies which even they coyly admit are “not strictly comparable”. They attribute their failure to find a correlation between inequality and mental illness in the USA to the fact that mental illness does not have a social gradient, but this does not deter them from reporting a correlation between inequality and mental illness on an international level.

They then mention, almost in passing, that rates of mental illness are evenly distributed between different races. In light of their failure to find a correlation with mental illness in US states, this should have been a Eureka moment but, as Saunders writes:

[T]hey fail to draw the obvious conclusion from their failure to find a relationship with inequality, which is that they only get state-level correlations with income inequality when there are underlying correlations with race to generate them. [emphasis in the original].

Since there is no relationship between race and mental health, they cannot find a relationship with inequality. But since there are relationships between race and many other criteria, they find correlations with inequality. But those correlations are statistical associations resulting from Wilkinson and Pickett’s failure to adjust for race. They are not causal. Inequality is a symptom, not the cause.

Wilkinson and Pickett never adequately address the question of causality. There are many important confounders such as income, race, education and material deprivation which are correlated with inequality, but are not caused by inequality. Conversely, many social problems such as crime, drug abuse and gang formation do cause inequality because young people growing up in environments with gangs, drug abuse and high levels of crime are less likely to succeed in society. We can address those issues by fostering job creation or crime reduction in neighborhoods with social problems. But, by Wilkinson and Pickett’s reckoning, inequality is the cause of these problems and not a symptom. This leads us to the improbable conclusion that societal malaise can be alleviated by reducing income in the surrounding neighbourhoods.

There is plenty of research—all of it ignored in The Spirit Level—showing that inequality does not have an independent effect on health and social problems once other variables have been controlled for. It should go without saying that countries differ from one another in many ways that have nothing to do with income inequality. That these differences will lead to different outcomes should be equally obvious. Wilkinson and Pickett justify their refusal to consider other variables in the postscript, saying “including factors that are unrelated to inequality, or to any particular problem, would simply create unnecessary ‘noise’ and be methodologically incorrect.”

With this one sentence, every historical, cultural, religious, political, legal, geographical, climatic and demographic difference between whole societies is dismissed as ‘noise’. Again, they are assuming that these factors are “unrelated to inequality” without putting that assumption to the test. It is no wonder Wilkinson and Pickett fail to identify confounding factors. They were simply not looking for them.