Lib Dem Voice

What does the data show about the Obama campaign’s use of data?

So far, there has been a weird paradox at the heart of the coverage about the Obama 2012 campaign machine. On the one hand we’re all meant to be impressed by how it was based on data and analysis, honing campaign techniques and targeting activity based on what the data said. On the other hand, we’re meant to take it all on trust (or trust plus bucketloads of anecdotes; i.e. trust) that this hard-nosed, evidence-based approach to campaigning worked. Where’s the evidence that the reliance on evidence really worked?

There’s been remarkably little presented. Which is why the following graph is so important.

It measures a very simple thing: how did the Obama 2012 vote share in each state compare to the Obama 2008 vote share? Overall, we know there was a small fall in his vote share. How did that play out across the different states? What variation was there between those states that had lots of TV ads from him, rallies a-plenty and all that high-tech, evidence-based campaigning and those states which his campaign basically ignored?

In other words, was there a uniform swing across the country or did his campaign do relatively better in the places which they targeted the most?

The answer: it was a uniform swing.

As the Larry Sabato’s Crystal Ball email newsletter, from which the graph comes, says:

How a state voted in 2008 was predictive of how it voted in 2012. The correlation between President Obama’s margin in 2012 and his margin in 2008 across all 50 states and D.C. is .96. In other words, you can closely predict Obama’s margin in 2012 almost perfectly from his margin in 2008; his drop from 2008 to 2012 was fairly uniform …

The biggest outliers are Utah, where Obama did substantially worse than expected in 2012, and Alaska, where he did substantially better than expected. Mitt Romney’s Mormonism probably explains why Obama underperformed in Utah, and Sarah Palin’s absence from the national ticket might explain Obama’s uptick in Alaska.

How to explain this? One option is simply to dismiss all the talk about the Obama campaign machine, put it down to excitable journalistic hype encouraged by the self-interest of the winners in talking up how great they were.

Certainly anyone who wants to argue that the Obama campaign really was good, needs to have a good answer to this.

An alternative is to credit the Romney campaign. Yes, the Obama campaign was great in the swing states – but in those it faced an intensive Romney campaign, so both could have been really effective – and cancelled each other out. There is a problem with that, as on most measures the Romney campaign seems to have been a lot less effective. Significant IT problems, deeply flawed polling, fewer Facebook fans, less impact on Twitter, not as many YouTube views; the list goes on.

The one measure on which the Romney campaign went head to head and achieved parity that I’ve found is TV advertising spend in the key states (though the Obama campaign generally claimed it got much better value for money). At best, then, the ‘but they were both great’ explanation becomes ‘sod the ground war, it was the TV advertising which mattered and they fought themselves to a draw’.

A more plausible explanation, based on the evidence so far, is that Obama ground campaign was very good but the ground campaign does not make that much of a difference when up against the state of the economy, a four year incumbency record and so on. The old rule of thumb often quoted in US politics is that the ground game can shift your vote by 1-3%. Perhaps for all the talk, that’s still the case. A slim difference like that would be vital in close races yet also get lost in the statistical noise of the 2008 to 2012 comparisons.

My guess is that as further evidence comes out it will be a two-part answer. One part – that American politics is so polarised socially, ethnically and geographically that there is much less room for clever campaign tactics to make a big difference than there is in the UK. The second part – that at the tactical level of techniques, there was plenty which was impressive even if in the end it didn’t alter Obama’s vote share much. With the greater ability to buck the national trend in the UK than in the US, the benefits of applying those tactical lessons in the UK will be all the greater.

However when someone from the US comes touting a lesson to learn, the first question back to them should be: look at the graph and tell me why your lesson is really true. Those with good lessons and genuine data will have a convincing answer to that. Those jumping on the latest hype bandwagon won’t.

UPDATE: Ethan Roeder, Director of Data for the Obama campaign, has written a response to this post.


Comment moderation policy