Political

Does it matter which parties are named by opinion pollsters in voting intention questions?

A frequently made, plausible-sounding and yet misplaced critique of voting intention opinion polls is over the choice of parties to ask about.

These polls list some parties up front and then give an ‘other’ option, behind which sits other, much smaller parties. Labour, for example, will be in the first category but the Women’s Equality Party in the second.

Which often leads people to complain that a poll is biased against party X because it is listed in the other section rather than in the main party listing. That sounds plausible: surely putting a party in the second division will depress its support, both because of the implicit label (hey, these are the less popular parties) and because picking one from the second list usually takes extra effort (such as in online polls where you have to pick ‘other’ and then go to an extra question about which other party)?

Plausible, but evidence has that annoying habit of popping up and telling us that what’s plausible and what’s true are not the same.

Here, for example, is BMG’s recent experience:

Since last month, BMG has reviewed the format of our vote intention question. Running a Random Control Trial (RCT) test, we showed half a representative sample the original version, and half the sample a version where all the parties featured in the initial list. The results showed only marginal differences for the Greens and the Brexit Party in terms of vote share received, all within the margin of error. Thus, including them in the “another party” list did not appear to be suppressing their support.

I’ve seen similar evidence on other occasions too.

Of course, because this is often untrue does not mean it is always untrue. But the evidence is a good reason to remember that going from ‘that sounds plausible’ to ‘and therefore the results are wrong’ can only be done if you’ve got some extra evidence to deploy.

Get polling news and analysis by email

Sign up here if you would like to receive the Polling UnPacked occasional email newsletter, highlighting the best in analysis and news about British political opinion polling from a carefully curated range of high-quality sites (no more than one email a day and usually much less frequent):

  • If you submit this form, your data will be used in line with the privacy policy here to update you on the topic(s) selected. This may including using this data to contact you via a variety of digital channels.
  • This field is for validation purposes and should be left unchanged.
Advertisements

There is one comment

Share your views

Your email address will not be published. Required fields are marked *

All comments and data you submit with them will be handled in line with the privacy and moderation policies.