What political science should learn from a diet book

Tim Spector’s Spoon-Fed (Amazon/Bookshop.org)* has for quite a while now been a best-selling book about what we eat. It has two main messages, one familiar and one that makes the book stand out.

The familiar one is about how fragile much of the research behind dietary advice is. Limited studies, small samples and questions about whether food industry funding has biased the research are all reasons to be cautious about many pieces of dietary advice. They are also the staple of many a piece of writing on food or on science.

But what Spector also adds is a second point, on the variability of humans. Suppose, let’s say, that research shows eat 5 grams of chocolate just before going to bed on average increases the number of minutes of deep sleep you get by 8%. The standard point is to doubt how solid that finding is; how many research projects really are behind that conclusion, is it based only on research on mice, was the research funded by Big Chocolate, and so on.

His second point, however, is that we are not all average. Even when the headline results are true, our variability makes them of limited benefit because it’ll often be the case that we fall a good way away from the average. It may be that even if the chocolate result is true, that you’re one of those who is far enough from the average on the downside that eating that chocolate has the opposite effect, and makes your sleep worse.

Or as Spector puts it in his conclusion:

[Once] we realise that any one of us is very unlikely to be average, it becomes obvious that attempting to follow prescriptive and ‘average’ guidelines, or someone else’s special diet, has a high chance of failure.

Delete person, insert country, and this is where political science comes in.

A good example of what I mean comes in the book The Politics of Competence. One of its findings is that when a government’s perceived competence goes up, so does its popularity.

But, the graphs in the book show a wide spread of data around this average finding. So even if the finding about the average effect is correct (and given the illustrious** nature of the authors and the book’s reception, I think it is), it’s not so useful as a guide to what a politician should do in a particular circumstance. On average, it may be true that raising its perceived competence raises a government’s popularity. But this particular government, in this particular country, in this particular year? As with Spector’s point about food, the graphs in the book show that there is such a spread of results, that the average being true isn’t that clear a guide to what will happen in any one particular case.

For pretty much all political science research that I come across, that caveat doesn’t seem to matter (i.e. the statistical analysis focuses on what the statistically significant pattern is, but then stops short of also looking at what the spread of reasonable expectations is around the average).***

Which is a shame because it greatly limits the applicability of such research for those who aren’t only reading about politics but are also in politics. For them, it matters whether, as with average diet advice, following average political science advice has a high chance of failure

* Affiliate links which generate a commission for each sale made.

** This judgement is, of course, independent from the fact they both follow me on Twitter.

*** I suspect some readers, whether illustrious authors or not, may bridle a little at this and be muttering comments about things such as how their tables show standard deviations. Standard deviations certainly feature in the sort of research I read, but my point is that the next step is almost always not taken – such as once you’ve decided what the average relationship is and that increasing X results in Y going up, going on to also say clearly how often the opposite effect occurs, that is how often increasing X results in Y going down. That’s the final step that would be really useful for those looking to use the work to inform what they do.

Sign up to get the latest news and analysis