Does it matter which parties are named by opinion pollsters in voting intention questions?

A frequently made, plausible-sounding and yet misplaced critique of voting intention opinion polls is over the choice of parties to ask about.

These polls list some parties up front and then give an ‘other’ option, behind which sits other, much smaller parties. Labour, for example, will be in the first category but the Women’s Equality Party in the second.

Which often leads people to complain that a poll is biased against party X because it is listed in the other section rather than in the main party listing. That sounds plausible: surely putting a party in the second division will depress its support, both because of the implicit label (hey, these are the less popular parties) and because picking one from the second list usually takes extra effort (such as in online polls where you have to pick ‘other’ and then go to an extra question about which other party)?

Plausible, but evidence has that annoying habit of popping up and telling us that what’s plausible and what’s true are not the same.

Here, for example, is BMG’s recent experience:

Since last month, BMG has reviewed the format of our vote intention question. Running a Random Control Trial (RCT) test, we showed half a representative sample the original version, and half the sample a version where all the parties featured in the initial list. The results showed only marginal differences for the Greens and the Brexit Party in terms of vote share received, all within the margin of error. Thus, including them in the “another party” list did not appear to be suppressing their support.

I’ve seen similar evidence on other occasions too.

Of course, because this is often untrue does not mean it is always untrue. But the evidence is a good reason to remember that going from ‘that sounds plausible’ to ‘and therefore the results are wrong’ can only be done if you’ve got some extra evidence to deploy.