What is it about?
We report the levels of heterogeneity, statistical power and research exaggeration found in a large survey of 12,000 psychological studies from 200 meta-analyses in nearly 8,000 articles. Typically, heterogeneity is high (74%), and only 8% are adequately powered. Heterogeneity this high fully explains the ongoing replication ‘crisis’. Researchers should change their expectations about the ease with which convincing evidence, either for or against an effect, can be claimed. Pre-Print: http://www.deakin.edu.au/__data/assets/pdf_file/0009/1459620/PrePrint-What-Meta-Analyses-Reveal.pdf
Featured Image
Why is it important?
Much has been said about the ongoing replication/credibility crisis. Our survey results explain why this crisis has happened and how some recommendations, such as greater data sharing, will have little effect on the probability of successful replication.
Perspectives
We were surprised to find only a small amount of average residual reporting bias even though 28% display a pattern consistent with publication bias. Regardless of selective reporting of statistically significant findings or other questionable research practices, the low power and high heterogeneity that our survey documents are sufficient to cause the observed low rates of successful replication.
Tom Stanley
Deakin University
Read the Original
This page is a summary of: What meta-analyses reveal about the replicability of psychological research., Psychological Bulletin, October 2018, American Psychological Association (APA),
DOI: 10.1037/bul0000169.
You can read the full text:
Resources
Contributors
The following have contributed to this page
