The failure of the polling industry to get the vote shares right in last week’s general election rightly means their methods are getting a post-mortem.
But there’s another form of political polling which is alive and well, yet deserves at least as much – if not more – scrutiny. It’s the polls that ask people why they behave the way they do and then treat the results as gospel.
More than a third of voters were influenced by the TV debates between the political leaders in the run-up to the election, a survey has found.
According to a Panelbase survey of 3,019 people, 38% were influenced by the debates, 23% by TV news coverage and 10% by party political broadcasts.
The kicker is the source for this: asking people what influenced them the most.
The problem with that is two-fold. First, it assumes people are honest in their answers. And second that, even if being honest, people understand fully what influences them.
Both are questionable – yet polling based on this pair of faulty assumptions is repeatedly rolled out as if it’s 100% useful.
The problems are best exemplified by the good looks of candidates. Experimental tests show that attractive candidates get more support (such as when people are giving mocked up information about candidates with just the photo changed). Yet do people go round telling pollsters they vote for a party depending on how attractive its leader is? They don’t.
Maybe they secretly know they’re not quite telling the truth. Or maybe they just aren’t that good at understanding exactly what motivates them to make their decisions.
Either way, simply asking voters isn’t enough unless you also buttress your results against those two fundamental problems – and that is almost never done.