What is it about?
In meta-analysis, a variety of statistical methods are used to estimate and analyze the information contained in some effect size index. In psychology, one of the two most used effect size indices is the standardized mean difference (Cohen's d), with its sampling variance being one of the essential elements for its analysis. There is a surprising variety of formulas to estimate this variance, which are also implemented in some of the most widely used softwares and packages. Some of these formulas exist since computers were not yet in common use for calculations and simpler formulas were needed, for use with a pocket calculator, but with approximate results. Others are simply wrong. Most meta-analyses based on d do not explicitly report the formula used for the sampling variance of d. Although the information on the software for the analysis gives clues about your choice, the calculations of d and its variance can be done with programs other than those for analysis and models fitting. In short, we rarely know what the chosen formula is and its justification. In this article we expose the most frequently used formulas, indicate software and packages in which they are implemented, and evaluate them through their bias, consistency and efficiency. We also show with numerical examples the consequences of the choice on the estimation of the other variance, the specific variance or variance of true effects, a crucial element to understand the relationships between the variables that are being studied. The article ends with some suggestions and recommendations.
Featured Image
Photo by Jean-Baptiste Lefevre on Unsplash
Why is it important?
The variety of formulas and the fact that it is often not reported makes it very difficult to replicate a meta-analysis. Sometimes it forces whoever tries to replicate it to deduce the formula used through the reported confidence intervals and the inverse calculation. In any case, it is important to be aware of the non-negligible consequences of the choice of formula, to put order in this part of the calculations, and to encourage greater unification in programs and packages for meta-analysis.
Perspectives
Writing this article has helped me put some order into a part of the meta-analytical calculations that always seemed confusing to me. I have had a great time discussing these ideas with my co-authors and other colleagues at conferences and meetings. I hope it helps to clarify the point of focus.
Dr Juan Botella
Autonomous University of Madrid
Read the Original
This page is a summary of: Methods for estimating the sampling variance of the standardized mean difference., Psychological Methods, December 2021, American Psychological Association (APA),
DOI: 10.1037/met0000446.
You can read the full text:
Contributors
The following have contributed to this page
