Are the Conservatives on 11% in Mid Bedfordshire?

I’ve written before about the misleading mystique of “internal polling” and this week The Sun gave us another great example with the story of another Mid Bedfordshire by-election poll:

Internal Tory polling has them on just 11 per cent in Mid Beds – putting them on course for one of the worst election defeats in history.

Big if true, as they say.

A clue however that the 11% may not be all it seems comes further down the story:

11 per cent of locals definitely plan to vote Tory at the next election.

To the general reader, that wording sounds pretty much the same as the quote. But to the trained eye, it’s stuffed full of clues that the first quote wasn’t all it seemed.

“Of locals” could mean that is 11% of everyone who took part in the poll, i.e. including those who said don’t know or wouldn’t vote or refused to say who they would vote for.

“Definitely” could mean excluding people who said, for example, that the would probably vote Conservative.

“Vote” could mean excluding people who said they weren’t 100% sure that they would vote at the by-election.

Let’s do a rough calculation to show what these three factors could add up to.

It’d be plausible for, say 40% of people polled to say don’t know/wouldn’t vote/won’t say (judging by other by-election polls). So if you exclude them, the 11% figure becomes 18% (11 out of 60 rather than 11 out of 100).

Then it’s plausible that, say, half the people who said they would vote Conservative are definite and half are likely, So that 11% turned 18% now becomes 36% picking Conservative.

Then we’ve got that turnout issue. When a pollster asks if you’re definitely going to vote, people tend to overreport their likelihood of voting. But also plenty of people who don’t say definite also do vote. So if you’re on 36% with just the definite to votes, it’s plausible that there’s half again support for you from the rest. Which would take the 36% up to 54% and the Conservatives on course to hold the seat comfortably.

To be clear, I doubt very much that’s what the poll really says. I think the truth is closer to the 11% than the 54%. and the Conservatives are in real danger in the by-election. But I also doubt that 11% is what you’d get for Conservative support from a properly done poll with all the stages completed for headline voting intention calculations.

I’ve also somewhat simplified the calculations that pollsters actually do to get their figures in my three point calculation above. The point, however, is to demonstrate the overall scale of such factors and so how limited the information is that we can draw from the story in the newspaper.

The mystique of leaked internal polling is replaced under careful analysis by doubt. With a proper poll and released data tables, the scale of the above three factors can be properly taken into account. With a story of a leak, we’re left not knowing where the truth lies in the polling numbers and with the nagging fear that perhaps if we knew the truth it’d be very different from the leak’s presentation.

The most interesting aspect of the story therefore is that it is The Sun, usually a Conservative supporting paper, which has decided to present the figure in a way that is so unflattering to Rishi Sunak’s government. Sometimes in the past we’ve seen right-leaning tabloid newspapers rallying around the Conservatives as an election nears. So far, that’s not happening (and perhaps for a reason so many went off the Conservatives in the 1990s – because their readers are being hit by the government’s economic failings).

Get the essential polling book