What does the data show about the Obama campaign’s use of data?

So far, there has been a weird paradox at the heart of the coverage about the Obama 2012 campaign machine. On the one hand we’re all meant to be impressed by how it was based on data and analysis, honing campaign techniques and targeting activity based on what the data said. On the other hand, we’re meant to take it all on trust (or trust plus bucketloads of anecdotes; i.e. trust) that this hard-nosed, evidence-based approach to campaigning worked. Where’s the evidence that the reliance on evidence really worked?

There’s been remarkably little presented. Which is why the following graph is so important.

It measures a very simple thing: how did the Obama 2012 vote share in each state compare to the Obama 2008 vote share? Overall, we know there was a small fall in his vote share. How did that play out across the different states? What variation was there between those states that had lots of TV ads from him, rallies a-plenty and all that high-tech, evidence-based campaigning and those states which his campaign basically ignored?

In other words, was there a uniform swing across the country or did his campaign do relatively better in the places which they targeted the most?

The answer: it was a uniform swing.
Graph showing the uniform swing in the 2012 US elections

As the Larry Sabato’s Crystal Ball email newsletter, from which the graph comes, says:

How a state voted in 2008 was predictive of how it voted in 2012. The correlation between President Obama’s margin in 2012 and his margin in 2008 across all 50 states and D.C. is .96. In other words, you can closely predict Obama’s margin in 2012 almost perfectly from his margin in 2008; his drop from 2008 to 2012 was fairly uniform …

The biggest outliers are Utah, where Obama did substantially worse than expected in 2012, and Alaska, where he did substantially better than expected. Mitt Romney’s Mormonism probably explains why Obama underperformed in Utah, and Sarah Palin’s absence from the national ticket might explain Obama’s uptick in Alaska.

How to explain this? One option is simply to dismiss all the talk about the Obama campaign machine, put it down to excitable journalistic hype encouraged by the self-interest of the winners in talking up how great they were.

Certainly anyone who wants to argue that the Obama campaign really was good, needs to have a good answer to this.

An alternative is to credit the Romney campaign. Yes, the Obama campaign was great in the swing states – but in those it faced an intensive Romney campaign, so both could have been really effective – and cancelled each other out. There is a problem with that, as on most measures the Romney campaign seems to have been a lot less effective. Significant IT problems, deeply flawed polling, fewer Facebook fans, less impact on Twitter, not as many YouTube views; the list goes on.

The one measure on which the Romney campaign went head to head and achieved parity that I’ve found is TV advertising spend in the key states (though the Obama campaign generally claimed it got much better value for money). At best, then, the ‘but they were both great’ explanation becomes ‘sod the ground war, it was the TV advertising which mattered and they fought themselves to a draw’.

A more plausible explanation, based on the evidence so far, is that Obama ground campaign was very good but the ground campaign does not make that much of a difference when up against the state of the economy, a four year incumbency record and so on. The old rule of thumb often quoted in US politics is that the ground game can shift your vote by 1-3%. Perhaps for all the talk, that’s still the case. A slim difference like that would be vital in close races yet also get lost in the statistical noise of the 2008 to 2012 comparisons.

My guess is that as further evidence comes out it will be a two-part answer. One part – that American politics is so polarised socially, ethnically and geographically that there is much less room for clever campaign tactics to make a big difference than there is in the UK. The second part – that at the tactical level of techniques, there was plenty which was impressive even if in the end it didn’t alter Obama’s vote share much. With the greater ability to buck the national trend in the UK than in the US, the benefits of applying those tactical lessons in the UK will be all the greater.

However when someone from the US comes touting a lesson to learn, the first question back to them should be: look at the graph and tell me why your lesson is really true. Those with good lessons and genuine data will have a convincing answer to that. Those jumping on the latest hype bandwagon won’t.

UPDATE: Ethan Roeder, Director of Data for the Obama campaign, has written a response to this post.

5 responses to “What does the data show about the Obama campaign’s use of data?”

  1. These results may also just indicate that US politics is highly tribal, with most voters already having made up their minds long before the campaign season. Do they do year-round campaigning in the US?

    • Andrew Suffield They do very little year-round campaigning compared to what we think of. However, they certainly do year-round fundraising, and when you add in the use of primaries plus the two yearly cycle of House contests, there's not that much of a quiet period anyway!

  2. Interesting. Could the answer lie in the fact that every state had senatorial and congressional races running concurrently? While the amount of effort put in by the Obama campaign in marginal presidential states will have to a certain extent been matched by additional effort from the Romney campaign, all Democratic campaigners have access to the data modelling and other techno wizardry invested in by the Obama campaign, no matter which state they live in. While the non-presidential campaigns in non-marginal Obama states won't have been so concerned with campaigning for Obama, the effort put in to turn out Democratic voters in a Senatorial or Congressional race, will have undoubtably benefited him anyway.

    • Tim Pollard Possibly, though it'd be quite an odd coincidence if the pattern of competitive state contests so neatly compliments the patterns of Presidential swing states so as to provide an even pattern overall. For example, California didn't get much Obama attention, but then the state wide races there weren't that competitive either and only a relatively small chunk of the area within the state was at lower levels. So how come California ended up with much the same swing as those swing Presidential states which also had competitive state-wide races?

  3. One point to bear in mind is that 4 of the 5 measures you claim show Romney's campaign was worse are internet/social networking site/.IT issues – a medium overwelminly favoured by the young who tend to be Democrats rather than Republicans.

Leave a Reply

Your email address will not be published.

All comments and data you submit with them will be handled in line with the privacy and moderation policies.