Why is a 1,000 sample enough for an opinion poll?
Even if you’re not scrabbling around for a reason to dismiss an opinion poll because you don’t like its findings, it can be surprising, even baffling, that a poll of only 1,000 people or so is meant to be enough to judge the views of a whole country.
How can that be? The mathematics behind sampling is old, well-established and straightforward enough to be taught in schools. But if you’re not familiar with it then, in my experience, it’s usually a more intuitive explanation that you’re after rather than a step-by-step explanation of the detailed maths.
Pioneering pollster George Gallup had a simple, powerful analogy. Imagine you have a bowl of soup in front of you. How much of the soup do you need to taste before you are confident what the whole bowl will taste like? Very little. Less than one complete spoonful, even.
Sure, you need to swirl around the soup, especially if it has pieces in it (the soup equivalent of random sampling). But then you can take just a tiny amount of soup and make a confident prediction about the taste of the whole bowl.
Adolphe Quetelet, the nineteenth-century inventor of the body mass index (BMI) and the concept of ‘the average person’, had a more upmarket version of this, although not developed about polling specifically:
Must I drink the whole bottle in order to judge the quality of the wine?
Returning to soup, now imagine replacing the soup bowl with a giant vat of soup. Once again, some vigorous swirling may be required. But once again, you will be confident you know what the soup is like from the tiny proportion you eat. Whether it is a bowl or a vat of soup you are tasting does not make much difference to how much you need
to eat to be confident you know what the soup is like.
It is the same with medicine. A blood test takes a tiny portion of your blood, yet those few drops are sufficient. Medical staff never say ‘we’ll take double the usual amount of blood today so that we can be really sure about the results.’
Which is why, just as you don’t worry about how little blood has been taken from you to test, you shouldn’t worry about a poll ‘only’ sampling 1,000 people. There’s plenty more that needs to be got right when doing a poll. But when done right, that is indeed enough to measure the views of the country.
To find out more about how opinion polls work, including what the margin of error on polls really means, see my new book out in April, Polling Unpacked: the history, uses and abuses of political opinion polls.
Hi,
A great analogy, but you should stress it only works wit5h RANDOM sampling. For example papers asking their readers to complete a survey is not random (those filling in the survey are self selecting, not to mention that it would only be the views of the subset of the population that reads that paper). Council officers often fall into this trap by posting a survey online and then publicising it via Twitter & Facebook. How often are councillors told that X percent of respondents to a recent survey supported change Y. As you say there is plenty more that needs to be ‘got right’ before a survey can be said to be representative.
Hi,
Take the European Referendum results and note the regional differences. Where does your 1,000 sample live and what are the local and geographical considerations?
I used to get my students to understand difference between the two main kinds of market research: Qualitative and Quantitive. They also had to understand the difference between Objective and Subjective. Soup? Taste can be very subjective.
It also depends on what questions the pollsters ask and what prompting they use. Anthony Wells of YouGov used to make this point on his, sadly now closed, blog. In particular, YouGov’s support figures for the Green Party are higher than for other pollsters because they prompt for them in their list of major parties, while other pollsters do not.