Between the 2017 and 2019 general elections, it became common for fans of Jeremy Corbyn to respond on social media to reports of a poor poll finding for their man with versions of ‘I’ll wait for Survation’.This was a reference to the polling firm Survation, which at the 2017 general election had stood out from the pack with its final pre-election poll being much more favourable to Labour than nearly all those from other pollsters. It had also been much closer to the actual result than those others.*
So a version of ‘I’ll wait for the pollster that was the best last time round, thank you very much’ was not, at first glance, such an unreasonable reaction to an unfavourable poll from other pollsters.
Except… give it a second glance and things look rather different.
Before we do that, let’s look at the origins of the phrase. There were a few tweets in 2017, after that year’s general election with Survation’s good polling performance, that pre-figured the later popularity of the phrase. That is shown by this search of public tweets that have not since been deleted or made private:
By late 2017 the use of the phrase was starting to be caricatured by others, with the earliest public example of this still available looking to be from December that year:
But it was in 2018 that the phrase, both used in earnest and in ridicule, really took off. It’s become such a meme that it’s even become a mug.
Yet through 2018, did it make sense for Corbyn supporters to unhappy non-Survation polls by waiting for the next poll from the pollster that had done so well at the last general election?
Not really, because you should take that second glance and ask, ‘Were Survation polls more favourable to Labour?’. Not really. During 2018 pollsters other than Survation averaged a 39% score for Labour, while Survation was nearly identical at 40%. Nor had Survation been painting a significantly more positive picture for Labour in 2017 between the general election and the end of the year. Again, Survation was but the one point different from everyone else with its Labour scores (43% to 42%).
‘I’ll wait for a poll that on average will be only 1 point better for my own side’ isn’t the most reassuring of responses to a poll whose findings you don’t like. Yet, ‘I’ll wait for Survation’ was used in earnest by people to dismiss polls that they wished were much more than just a single point off.
It make this a powerful example of what often happens when people try to be clever about polls. Someone makes an apparently plausible claim about how to interpret polls or how to pick which ones to pay attention to, but then doesn’t take the next step of asking ‘if my claim makes sense, what evidence would there be to back it up?’
Hence people dismissing non-Survation polls that were not much different from Survation polls. Or indeed people dismissing YouGov polls as favouring the Conservatives despite the evidence again showing otherwise.
Though both of these examples of failed attempted polling smarts come from people on the left, there are plenty of examples where it’s people from other political perspectives making similar errors. Nor is it restricted to the UK, as the example of Republicans earnestly ‘unskewing’ polls to show how Romney was going to beat Obama in 2012 demonstrated. (The polls weren’t wrong, Mitt Romney didn’t win.)
Which is why that second glance is a good rule of thumb for us all to remember.
LIKE TO KNOW MORE ABOUT HOW TO MAKE SENSE OF THE POLLS? Check out my new book, Polling Unpacked: the history, uses and abuses of political opinion polls.
* The Conservatives won the 2017 election with a 2% lead in the popular vote. Survation’s last poll had put it at 1%, much more accurate than BMG (13%), ICM (12%), ComRes (10%), Ipsos MORI (8%), Panelbase (8%), Opinium (7%), YouGov (7%) and Kantar-TNS (5%). Only two low profile pollsters were closer, but still not as accurate as Survation – Qriosly which had Labour 2% ahead and SurveyMonkey which had the Conservatives 4% ahead. Note, by the way, how despite the popularity of claims about YouGov bias, it was far from being the worst in overstating the Conservative lead.