Quantitative Data From Rating Scales: An Epistemological and Methodological Enquiry

Jana Uher
  • Frontiers in Psychology, December 2018, Frontiers Media SA
  • DOI: 10.3389/fpsyg.2018.02599

Numbers from ratings: Scientists' measurement concepts do not match what raters think

What is it about?

What is going on in raters' minds while ticking scales is very different from scientists' concepts of scientific measurement and quantification. Scientists' high theoretical aspirations for quantifying behaviour, mind and society place high demands on raters who are given a tool that does not allow them to fulfill this complex task. Instead, raters use mental shortcuts and focus on concrete examples that they can remember to make their judgements.

Why is it important?

When answering rating scales, raters have to make sense of the question at hand and come up with concrete examples to which they refer to make their judgement. The article shows that there are pronounced differences among people in the ways in which they do that. In fact, they often think about very different things when answering the same question. But scientists commonly ignore this diversity - in fact, they hardly ever ask! Strikingly, most raters do not have any quantitative ideas in mind when ticking rating scales - and yet researchers convert these ticks into real numbers and then treat them as scientific quantifications.

Perspectives

Dr Jana Uher
University of Greenwich

Important decisions are made on the basis of rating data, such as for individuals' careers and national policies. Given this, scientists must urgently explore how raters actually understand questionnaires, how they generate their answers to standardised scales and what they actually want to say about the issues at hand.

Read Publication

http://dx.doi.org/10.3389/fpsyg.2018.02599

The following have contributed to this page: Dr Jana Uher