What is pre-registration?

+4 votes
asked Aug 10, 2015 in Open Science by Rex Kerr (95 points)

Pre-registration seems to have something to do with stating how one is going to test hypotheses before one collects data.

Is this all there is to it?

Where can one find a good defence of this being a good idea? It of course helps openness, but does it help quality of science also?

Research often delivers surprising results--correlations one hadn't thought to look for before collecting the data. How does pre-registration accommodate serendipitous discovery of this sort?

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)
commented Aug 18, 2015 by Alexander Konovalov (135 points)
If you think that this thread should be migrated to Academia or another SE site because the OpenScience beta is closing, please edit the list of questions shortlisted for the migration [here](http://meta.openscience.stackexchange.com/questions/73/).

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)

1 Answer

+6 votes
answered Aug 10, 2015 by Tom Hardwicke (155 points)
Best answer

Conducting a pre-registration typically involves specifying your research design, data collection procedures, and data-analysis decisions prior to beginning your study. Here is an example.

The problem that pre-registration seeks to address is as follows: in any given research study there are numerous potential research decisions that can be made. For example "When should data collection be halted?" or "How should the dependent variables be defined?". These different decision paths are often referred to as researcher degrees of freedom. The issue here is that exploiting researcher degrees of freedom (e.g., trying many decisions paths until one reaches a favoured result) can easily lead to false-positives.

The purpose of pre-registration is to make transparent which research decisions were made before conducting the study, and which were made during or after conducting the study. When a paper is published, the pre-registration document can be used to make this distinction, which are also referred to as exploratory vs confirmatory findings.

Note that pre-registration does not prevent the reporting of exploratory findings, it only makes transparent which findings are exploratory and which are confirmatory. For example, you may realise after conducting your study that a different analysis technique, which you did not mention in your pre-registration, could reveal something interesting in your data. You could still use this technique and report the findings - all pre-registration requires is that you make transparent that this decision was made after rather than before data collection began. The reader can then make an informed evaluation of the evidence being presented.

For arguments in favour of pre-registration, this article "An Agenda for Confirmatory Research" is a great place to start.

There are many different approaches to conducting pre-registration in practice. One informal method is to upload your pre-registration document to a public 3rd party repository, such as the Open Science Framework. A more formal approach is Registered Reports - a type of publication format in which the pre-registration document is submitted to a journal, reviewed, and, if of appropriate quality, the study is accepted for publication before the data has been collected. Registered Reports therefore, has the additional potential benefit of helping to correct for publication bias.

This article is a very informative introduction to Registered Reports and pre-registration in general.

Finally, if you would like to give pre-registration a try for the first time then you might be interested in the forthcoming Pre-Registration Challenge being run by The Center for Open Science. You will be walked through the process of pre-registration and there is a potential $1000 incentive. This large-scale trial of pre-registration will be evaulated by a parallel meta-science project that will evaulate its effectiveness and practical utility.

This post has been migrated from the Open Science private beta at StackExchange (A51.SE)

Welcome to Open Science Q&A, where you can ask questions and receive answers from other members of the community.

If you participated in the Open Science beta at StackExchange, please reclaim your user account now – it's already here!

e-mail the webmaster