Five tips to make sense of voting intention opinion polls

Beware random fluctuations that ‘plunge’, ‘soar’ or ‘collapse’ in a headline. Instead, track trends over time.

The volume of opinion polls hit record levels in 2009 and has continued to go up.

So how to make sense of the blizzard of numbers? A good place to look to get a full picture of the polls is Anthony Wells’s UK Polling Report. But here are some pointers.

Rule one: carrying out an opinion poll is like flipping a coin. Flip a coin 10 times. Then do it another 10 times. And again. Chances are the number of heads each time will vary. But that doesn’t mean someone’s swapped the coin for a loaded one in between. Instead, it just reflects random fluctuations between each set of flips.

It’s the same with polls: you get random fluctuations even if a party’s support has not actually changed.

Rule two: journalists almost never say, “True figures could be up, could be down, could be the same. We just don’t know.” Those random fluctuations almost always get written up as “soared”, “plunged”, “surged” and “collapsed”.

Rule three: almost no changes between two individual polls are statistically significant – in other words, big enough to be sure that it’s not just the randomness at work. Because one poll may be on the high side and one on the low, it’s only changes of around 5% (more than the margin of error on an individual poll) in a party’s support between two polls that are statistically significant – and that size change almost never happens.

Rule four: you need to look at the trend across several polls. All the changes may be within the margin of error on themselves, but the pattern may be clear and significant. When looking at the trend, make sure to compare like with like: ICM with ICM, YouGov with YouGov, etc, as different pollsters do have some systematic differences.

Rule five: ignore which media outlet a poll appeared in. A Mori poll is a Mori poll, regardless of who commissioned it. Usually, media outlets (with the honourable exception, generally, of the Guardian) only compare their poll with the previous one they paid for – even if there was another poll from the same company, carried out in just the same way, in between.