Political

It’s not only medicine that has issues with non-publication of trial data

Welcome to the latest in my occasional series highlighting interesting findings from academic research. Today – the problem of ‘uninteresting’ trial data not being published.

This has become a high profile issue in medical research, with the concerns that trial data where a treatment is found to have no impact is much less likely to be published than trial data which does show an impact. The result? The published data may give a very lopsided view of what is really happening.

This is not just an issue for medicine, however, as Donald Green, Mary McGrath and Peter Aronow have pointed out in Field Experiments and the Study of Voter Turnout (Journal of Elections, Public Opinion and Parties, 2013, Vol.23, No.1).

Gathering data on political science field experiments designed to test the impact of different ways of raising turnout (e.g. sending letters, making phone calls, and so on), they found that:

The main challenge is the ‘file-drawer problem’ whereby experiments with null findings go unreported. We have made a concerted effort to compile a comprehensive set of studies, including unpublished studies … As expected, unpublished studies tend to have smaller effect sizes than published studies.

One response to “It’s not only medicine that has issues with non-publication of trial data”

  1. It's a real problem, but the cause isn't just academics shoving stuff into file drawers (although I'm sure that goes on). It's that getting stuff published in journals is much harder if it's got a null finding — I speak as someone who's got a paper with a null finding that has been pinging back rejected from journals for well over a year. (Of course, it might also be crap, but I am choosing to blame the null finding…). And so after a bit, you give up and move into the next project.

Leave a Reply

Your email address will not be published. Required fields are marked *

All comments and data you submit with them will be handled in line with the privacy and moderation policies.