political

2015 opinion poll error same as the one that caused Lib Dems to be over-estimated in 2010

It had always seemed to me likely that what went wrong with the polls in 2015 had a heritage in 2010, in that in both cases the polls over-estimated the party which seemed to be doing best at appealing to younger, lower turnout voters (Labour in 2015, Lib Dems in 2010). Yet the real experts crunching through the evidence from 2015 have – until now – pretty much ignored the possible 2010 link.

Which makes this press release today very interesting:

BES reveals longstanding flaw in opinion polls

Flaws which made the opinion polls wrong in 2015 also explain errors in 2010, new research from the British Election Study (BES) reveals.

BES researchers Dr Chris Prosser from The University of Manchester and Dr Jon Mellon from The University of Oxford last year revealed that sampling was the reason for the 2015 polling miss, confirmed in Tuesday’s Independent Inquiry preliminary report.

Now Drs Mellon and Prosser show that the same mechanism of over-representing groups such as young people who were less likely to vote, is also likely to have been at least partially responsible for the 2010 over-estimation of the Liberal Democrats’ vote share.

The flaw, they say, was also likely to have been present in the 2001 and 2005 general elections, but the electoral maths meant the polls correctly forecast which party would form the Government, despite overestimating Labour’s share.

Dr Mellon said, “We have shown the 2015 polls were fundamentally unrepresentative of the electorate and that Labour leaning respondents were the most over-represented as voters in the sample.

“Our new research shows that this problem is a longstanding issue: in 2010 these over-represented respondents were Liberal Democrat voters, which could potentially explain the 2010 polls’ over-estimation.”

In 2010, the polls over-estimated the Liberal Democrats by around 4.0 percentage points, but no reason was found for the error.

The BES findings now suggest the error was down to sampling voters from demographic groups with low turnout demographics, who tended to support the Liberal Democrats.

Dr Chris Prosser said: “The surprising surge of support for the Liberal Democrats among low-turnout demographic groups, such as students, may in fact have saved the pollsters from a 2015 polling miss in 2010.”

The academics have devised a way of correcting the flaw, so pollsters can in future create more accurate forecasts by factoring in additional weighting to account for expected proportions of non-voters.

By adding an additional weighting factor of the expected proportion of non-voters, respondents from demographics which vote less would be more represented in the non-voting section of the general population sample.

That means they will be less represented as voters – improving accuracy.

“While we do not know exactly what the turnout will be in a given election, we do know that it is likely to be reasonably close to the previous election’s turnout,” said Dr Mellon.

“But further work is needed to determine whether this method will be sufficient.”

The BES face-to-face survey, carried out by GfK between May 2015 and September 2015, was a random probability sample of 2,987 people.

It showed a closer match to the 2015 general election Conservative lead over Labour than the pre-election polls, which failed to forecast the scale of the Conservative victory.

Analysis of the data by the BES team suggests that this closer match was made possible by using probability-based sampling methods and successful targeting of hard-to-reach groups, including non-voters.

Dr Mellon said: “We believe these findings have serious implications for pollsters both in the UK and in other countries.

“Weighting to population targets and then removing non-voters is by far the most dominant election polling approach, and any pollster that uses it risks the same kind of polling miss that Britain saw in 2015.

“Given the increasing number of polling misses throughout the world, we may already be seeing the results of this methodological choice.”

As for the question of what went wrong with the Liberal Democrat constituency polling, see my earlier post.

2 comments
John Napper
John Napper

So those who don't vote are more likely to not vote Lib Dem. 

That actually makes sense as far as the incorrect opinion polls are concerned. I'm not sure it helps anything else though.

One thing that you can take from this is that actually persuading these people to vote is more likely to benefit the Lib Dems. The tricky thing is how to do that. Jeremy Corbyn seems to be making headway amongst the disillusioned so that is a clue. They don't think there is any point in voting for more of the same whatever the party is called. There has to be something clearly different and worth voting for. It doesn't have to be left-wing, just different and positive.

Now the really difficult thing is overcoming the age-old third party conundrum of actually persuading people that their vote won't be wasted. The old "I would vote for you if I thought you had a chance" argument where it is very difficult to get the message across that if everybody who thought that way actually did vote Lib Dem (or Green or whatever party they say this to) then maybe the candidate would have a chance. It's the perception of a 2 party system when it comes to who forms the government which has to be overcome.

The Lib Dems came close to breaking that mould but Clegg and the coalition blew it and now we have to start again. So far Tim Farron is coming across as Blair ultra-light and that is the road to nowhere. It's no good aiming at pleasing the party faithful because you already have their votes. It's the people who switched their vote from the Lib Dems, those who stopped voting together and those who have yet to vote but just might if they saw something they thought was worth the effort that you have to aim to attract.

trevdick
trevdick

This emphasises the need for the party to have clear, coherent policies aimed at Older voters. There is plenty of scope! Elderly Care,Pensions, Frozen Overseas Pensions, Age discrimination in access to services, particularly Health Care, Elder abuse, Care in Community...the list goes on!

Trackbacks

Comment moderation policy