Critical debates on fundamental problems in quantitative psychology and measurement (Special Issue)

What is it about?

This Special Issue presents 15 original articles that critically explore fundamental problems underlying psychology's crises in the replicability, validity, generalisability of its findings. The 38 authors scrutinise established theories and practices that are widely used in quantitative psychology and psychological ‘measurement' and open up novel and revived perspectives on fundamental problems. Uher, Arnulf, Barrett, Heene, Heine, Martin, Mazur, McGann, Mislevy, Speelmann, Toomela, & Weber argue that psychology's crises are rooted in the Questionable Research Fundamentals (QRFs) of many of its theories, concepts, approaches and methods (e.g., of psychometrics)—and therefore cannot be tackled by just remedying Questionable Research Practices (QRPs) as currently believed. Luchetti explains the problem of epistemic circularity that is inherent to measurement and analyses how Fechner tackled this problem in psychophysics. Kuhbandner and Mayrhofer evaluate limitations of experimental psychology, highlighting that even minimal differences in the experimental setup considered irrelevant can build up to large unsystematic effects. Mayrhofer, Büchner and Hevesi interpret the replication crisis primarily as a symptom of an epistemological crisis derived from the mismatch of psychology's quantitative methods with the nature of the psyche. Linkov argues that pure (‘qualitative') mathematics, the study of mathematical concepts, could help scientists to distinguish more clearly between qualitative pattern descriptions, quantification and numbers as well as to tackle the ensuing challenges to understanding measurement. Scharaschkin demonstrates a ‘qualitative' mathematical theorisation for educational assessments (e.g., learner proficiency), highlighting that valid representations are theory dependent and need not be conceptualised structurally as values of quantities. Scholz proposes Barad's agential realism as a suitable alternative philosophy of science for quantitative psychology, which requires reconceptualising common assumptions about ‘true scores', the research process and researchers' independence of their research objects. Ramminger and Jacobs discuss the critical role of theory in understanding and evaluating validity in psychological ‘measurement', contrasting three different positions on validity. Uher provides a comprehensive critique of psychology's overreliance on statistical modelling at the expense of epistemologically grounded measurement processes, showing why the inbuilt semantics of language-based methods leads many to mistake judgements of verbal statements for measurements of the phenomena described. Arnulf, Olsson and Nimon analyse how and why digital language processing can predict psychometric and statistical results fairly accurately even without access to human response data, highlighting that psychology often constructs only semantic rather than nomological networks as commonly assumed. Hanfstingl, Oberleiter, Pietschnig, Tran and Voracek emphasise the importance of identifying jingle and jangle fallacies and propose a four-step procedure to detect and address issues related to these fallacies. Slaney, Graham, Dhillon and Hohn explore the rhetorical language commonly used in scientific discourse about the theory, validity and practice of psychological ‘measurement' and identify relevant themes, illustrated with constructive and useful but also misleading and potentially harmful discourse practices. Reisenzein and Junge introduce a framework to study the intensity of emotions that bridges theoretical assumptions and empirical methodologies and offers insights for improving the precision of emotion-related assessments. Brauner proposes to study not 'latent constructs' but instead several, disparate assessment points in so-called ‘micro scenarios' as an integrative contextual method to evaluate mental models and public opinion. Paredes and Carré emphasise the necessity to develop a wider and more nuanced understanding of how different people, communities and cultures interpret and use psychometric ‘scales' using participatory approaches.

Featured Image

Photo by Peter Kasprzyk on Unsplash

Photo by Peter Kasprzyk on Unsplash

Why is it important?

Science lives from controversy and the diversity of perspectives. This is a prerequisite for new developments.

Perspectives