What is it about?
Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate.
Photo by JOSHUA COLEMAN on Unsplash
Why is it important?
The multiple testing of hypotheses occurs in the most areas of science. For example, it occurs in clinical science, where researchers investigate whether a treatment affects multiple disease symptoms, and it occurs in psychology, where researchers investigate whether multiple groups of people hold different attitudes to one another. Multiple testing has been implicated in the replication crisis in science. In particular, it has been suggested that researchers who do not adequately correct their significance threshold, or alpha level, during multiple testing are at a greater risk of making Type I errors (incorrectly rejecting null hypotheses) and, consequently, publishing nonreplicable false positive results.
Read the Original
This page is a summary of: When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing, Synthese, July 2021, Springer Science + Business Media,
You can read the full text:
The following have contributed to this page