What is it about?

Reporting of reliability information of measures on the studied sample is a basic requirement in psychological research. This paper identifies limitations of an existing method for computing reliability with multilevel data---data that have a hierarchical structure such as students nested within schools and survey participants nested within neighborhoods, and proposes alternative indices that address these limitations.

Featured Image

Why is it important?

This paper shows that the previously proposed between-level composite reliability can provide overly optimistic reliability coefficient because it ignores one major source of error, namely the sampling error of cluster means. To obtain more accurate reliability information for multilevel data, this article proposes alternative reliability indices that correctly account for the different sources of measurement error. The computation of the between-level indices also takes into account whether the construct being measured is an aggregate of individual characteristics (e.g., mean student achievement of a school) or an inherent group-level characteristic (e.g., school climate). I illustrate the proposed indices using a large-scale national data set with four items measuring students' attitudes toward mathematics. Software code in the R and the Mplus software was provided so that applied researchers can use to compute the proposed indices for their data and the corresponding confidence intervals.

Perspectives

I got interested into this topic when writing a section to discuss reliability for a book chapter about multilevel confirmatory factor analysis. Writing this paper is a great exercise for me to understand what psychological constructs mean at an individual level and at the group level.

Hok Chio Lai
University of Southern California

Read the Original

This page is a summary of: Composite reliability of multilevel data: It’s about observed scores and construct meanings., Psychological Methods, February 2021, American Psychological Association (APA), DOI: 10.1037/met0000287.
You can read the full text:

Read

Contributors

The following have contributed to this page