What does the polling say about a possible Progressive Alliance?

Last month a story did the rounds about how a poll shows that if Labour, Greens and the Liberal Democrats did a deal to stand only one candidate against the Conservatives, Boris Johnson would be out of power.

But how good was that poll and what conclusions can really be drawn from it? Answering that question is important in its own right, but I’m also going to show the steps to the answer – showing how to approach evaluating an opinion poll finding that is in the news. (More on all that in my book, Bad News: what the headlines don’t tell us.)

Reputable pollster?

The poll was conducted Find Out Now. They are a member of the British Polling Council, the self-regulatory body for British political poling and the full data tables are available online. So far, so good.

The right methodology for the task

The poll was an MRP poll, which allows constituency level results to be extrapolated from large national samples. That’s a good methodology to use to answer questions like ‘how would a seat pact work?’ as you need to get into how results might vary between different types of seats.

But MRP polls should come with a big caveat. Basic opinion polling is a fairly widespread skill, with plenty of experts and long track records. MRP is not. MRP methodology is complicated, with only a small number of real experts and with a limited track record. It is also much more of a ‘black box’ than more traditional polling.

All of this means the only real judgement of how good a particular team’s MRP model is comes when you have actual results to compare them against.

Pollster track record

Find Out Now teamed up with Electoral Calculus for this MRP poll and there isn’t much of an MRP track record for them. The 2019 general election prediction from Electoral Calculus was respectable, but it wasn’t a pure MRP prediction. Rather, it included other data, such as polling averages.

There was a MRP poll for the 2021 local elections but its accuracy is far from clear. The seat predictions were way off, e.g. predicting Labour to gain 280 seats when in reality Labour lost 326. However, the seat numbers were all rather strange and didn’t seem to even add up to the right total number of seats (e.g. see the replies in this Twitter thread). Electoral Calculus’s own published assessment of this prediction simply leaves out the seat numbers and only talks about council control and vote share. On those two measures, it did well. But on seat numbers, something went wrong.

The actual questions asked

Turning to the poll specifically, it was based on working out a possible set of seat deals, picking one of Labour, Green and Lib Dem to stand in each seat and asking people in England and Wales (though not Scotland) how they would vote. People were given the choice of candidates appropriate for their own seat:

Q2. Suppose at the next general election that all the usual political parties are standing in your seat except that the [OTHER PARTY1] and [OTHER PARTY2] have agreed not to stand and are asking their supporters to vote [SELECTED PARTY]. Which party, if any, would you vote for, in this general election?

[SELECTED PARTY, OTHER PARTY1, OTHER PARTY2 are “Labour”, “Liberal Democrats”, “Green party” depending on respondent’s seat]

The results were then fed into an MRP calculation to provide seat numbers. These were compared with seat numbers without a pact, apparently showing a big boost for having such a pact with Labour wining 36 seats more with a pact, the Lib Dems winning 14 more and the Greens 8 more. Those 58 gains remove the Conservative majority.

But the with/without pact comparison was not a like for like one.

The poll didn’t ask a normal voting intention question and then a pact question. Rather, it just asked the pact question and compared the results from that with the results from a poll of polls, looking at standard national voting intentions.

So although the MRP pact poll finds a difference from traditional voting intention polls, we don’t know how much of that difference is due to the former being an MRP poll and how much due to the theoretical pact. If both MRP and other polls are perfect (or equally in error!) then that doesn’t matter. But we’ve no way of knowing if that is the case, which means the supposed impact of the seat pact is left uncertain.

It’s a real shame that the MRP poll didn’t ask both voting intention questions and so give a much better like-for-like comparison.

Theory and reality

However, even if it had, there’s still a problem about the question asking voters a hypothetical: if X happens, what will you do? There’s a risk that people may not predict their behaviour accurately. There’s also a risk that by just asking about X a poll question focuses people’s attention on it much more than you would get in reality.

Take the YouGov poll from 2016 which I reported as ‘New poll shows Lib Dems could overtake Labour, Ukip if politics polarises over Brexit’. We got an election in 2019 that was rather like some of the scenarios in that poll but a result that was very much not like its findings. The poll was right in the sense of showing that Brexit might fuel a powerful Lib Dem performance. Might rather than would because other things can – and did – happen.

So even if the MRP poll on seat pacts was perfect, it would still suffer from this forecasting problem.

Those seat deals in detail

Before concluding what we can learn from this poll, there’s another detail that needs examining. The poll was based on a theoretical set of seat deals. Some of those deals (see Appendix 2 in the report) were very unusual.

Take Lewes, where the result in 2019 was:

Conservative 48%
Liberal Democrat 43%
Labour 6%
Green 3%

The deal the poll was based on? Labour and the Liberal Democrats standing down for the Greens in this seat (!). Which highlights a problem: there are no Conservative-held seats in which the Greens are in a strong second place as the obvious challenger. So any sort of seat deal would be very difficult to agree, as it either involves other parties standing aside where they can legitimately think they have a better chance than the Greens, or it involves the Greens standing aside without getting much in return.

The Conservatives won’t do nothing

The Lewes example also shows one other risk. At 48% the Conservatives were not far short of an absolute majority in the seat anyway.

In practice, what would happen with a seat deal? The Conservatives would use it as a political weapon to win over support – e.g. by squeezing party X’s old support by saying a vote for X is really a vote for Y, and also be using it as a motivator for its own supporters – ‘you’ve got to go vote to stop the other parties conspiring against us’.

A poll about a theory can’t capture well the reality of the counter-reaction if theory were to become reality.

What can be concluded?

Where does this all leave us? The polls gives a clue that Green, Labour and Liberal Democrat supporters may be willing to switch to defeat the Conservatives.

It’s a tentative clue as it doesn’t give a clear like-for-like comparison between seat deal and no seat deal. It’s also a tentative clue as it doesn’t capture the political dynamics and difficulties that would be involved in any such deal. Nor did it include Scotland.

So it’s a clue that suggest there’s a willingness to switch. But it doesn’t go so far as to tell us the best way to make that sort of switching happen, i.e. it doesn’t tell us enough to judge the relative merits of formal seat deals versus a more informal targeting of different campaigning and encouragement of tactical voting, 1990s-style.

A piece of evidence, then, rather than an argument clincher.

9 responses to “What does the polling say about a possible Progressive Alliance?”

  1. Labour, in Wales, gives only one reason for voting Labour: To Keep the Tories Out. It’s even their message in places where there’s never ever been a Tory candidate elected at any level and it results in places like Aberafan having Kinnock as their MP and Tory policies filtering from the Welsh Labour government and the Labour councils.

  2. All this assumes that everybody who voted L / LD / G would vote for an Alliance candidate.

    I contend that this is a risky and probably erroneous assumption. Some might vote C because they resent being cheated of their preferred party. More might abstain. Other credible candidates might stand and take away votes for the Alliance. Turnout in those circumstances would be critical.

    That’s even without triangulating for the previous voting records of the respondents or taking into account the time period in which the question was asked or the likely date of the election.

    I suspect that if the Batley and Spen poll had taken place a fortnight earlier or later the Tories would have won handily.

    • As a general point, I agree with you John. There are too many assumptions that all the votes for X will transfer to Y. For this particular post, though, it’s worth emphasising that it’s about a poll which asked people what they’d do. So that particular criticism isn’t applicable to it.

  3. “The seat predictions were way off, e.g. predicting Labour to gain 280 seats when in reality Labour 326. ”

    Eh? [See above]

  4. The point surely is that it is obvious in most seats whether Lib Dem or Lab are next placed to beat the Tories.
    The Greens would have very few seats to fight but they could be persuaded to stand down on the basis that a coalition government would commit to introduce PR. Simple!

  5. this would be worse than ‘taking the voter for granted’, it’s ‘treating the voters like sheep’, expecting them all to be herded into support of the chosen one.
    Trying to predict an outcome on this is rather like trying to predict a PR vote on the stats of a FPTP vote, it doesn’t work, because when you change something the voter attitude changes. Far better to have a cross-party alliance on voting reform policy, and publicise that in the run-up, and then leave the voter to vote tactically.. they will.!

  6. I’m really pleased to see Mark Pack seriously chewing over the evidence for a Progressive Alliance and concluding that the Find Out Now poll is, on balance, another piece of evidence in its favour. We need to see our leading lights working out how to make a Progressive Alliance successful, for the whole country’s benefit.

    And I hadn’t realised before what a zero-sum game this would be for the Greens, with them not being a major challenger in any Tory-held seat.

    Which are the 10 (say) Tory seats where the Greens could best mount a serious challenge? If the Progressive Alliance identified them, and earmarked them as Green territory, that alone would have a significant effect. But seats such as Lewes, a former LibDem hold, or recent Labour holds for that matter, should not be on the list.

    Nobody says negotiating a PA will be easy. But success in negotiating it will be a triumph in itself.

  7. The question asked by the pollsters was completely unrealistic. A lot of people are unhappy with the Conservative government, but they don’t necessarily think a Labour one, even in coalition with the LDs and Greens, would be any better. They dislike intensely the power that the trades unions have over Labour through their financial support. The only good reason for forming an alliance of the three parties is to get a Commons that will push through electoral reform. For that purpose the SNP should be a willing fourth partner. Before asking the public what they think about an alliance, the allies must formulate and publish their strategy for what they would do with a Commons majority. This must be essentially to introduce legislation within a few weeks for a specific sufficiently proportional electoral system already agreed upon – STV is the LDs choice, and mine – and to call a further general election under the new system as soon as possible after that has been enacted. Meanwhile the allies should commit to avoid introducing any other new legislation unless it had the backing of all of them. It would be essentially a caretaker government on all matters other than getting PR (as defined) done. Postulating a cross-party alliance is completely pointless without also spelling out in detail what that alliance would and would not actually do. Cross-party discussions on that should be underway already, and concluded within 12 months.

Leave a Reply

Your email address will not be published.

All comments and data you submit with them will be handled in line with the privacy and moderation policies.