What is it about?

Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis tests, which involve planned tests of ante hoc hypotheses, and exploratory hypothesis tests, which involve unplanned tests of post hoc hypotheses. This distinction is thought to be important because it has been proposed that confirmatory hypothesis tests provide more compelling results (less uncertain, less tentative, less open to bias) than exploratory hypothesis tests. In this article, we challenge this proposition and argue that there are several advantages of exploratory hypothesis tests that can make their results more compelling than those of confirmatory hypothesis tests. We also consider some potential disadvantages of exploratory hypothesis tests and conclude that their advantages can outweigh the disadvantages. We conclude that exploratory hypothesis tests avoid researcher commitment and researcher prophecy biases, reduce the probability of data fraud, are more appropriate in the context of unplanned deviations, facilitate inference to the best explanation, and allow peer reviewers to make additional contributions at the data analysis stage. In contrast, confirmatory hypothesis tests may lead to an inappropriate level of confidence in research conclusions, less appropriate analyses in the context of unplanned deviations, and greater bias and errors in theoretical inferences.

Featured Image

Why is it important?

It has been argued that the replication crisis in science may be related to people's overconfidence in the replicability of results caused by the false portrayal of exploratory hypothesis tests as confirmatory hypothesis tests. New research practices such as preregistration are supposed to address this potential problem.

Perspectives

In our view, the causes of the replication crisis may be entirely unrelated to the confirmatory-exploratory distinction. For example, the crisis may have been caused by a base rate fallacy, heterogenous effects, poor validity, low power, and/or hidden moderators. A lack of attention to factors such as these may have led to overconfidence in replicability independent from the confirmatory-exploratory distinction.

Prof Mark Rubin
Durham University

Read the Original

This page is a summary of: Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests, Philosophical Psychology, August 2022, Taylor & Francis, DOI: 10.1080/09515089.2022.2113771.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page