What is it about?
Essays are mainly marked by human raters who bring their own subjective criteria, which can lead to discrepancies and raise issues of reliability and fairness. This study explored the criteria used by markers in a national high-stakes 12th-grade examination run by three examination boards in the south of Pakistan. Fifteen markers scored the same three essays without a rating scale, as in current practice, and also took part in interviews and wrote short comments justifying their scores.
Featured Image
Photo by Aaron Burden on Unsplash
Why is it important?
Many-facet Rasch analyses showed differences in markers' consistency and severity, and the interviews and comments revealed great variability in their criteria: grammar, attitude towards mistakes, handwriting, length, creativity, organisation and use of cohesive devices. Even when markers apply the same criteria, they give them different weight, which has implications for the fairness of essay scoring.
Perspectives
Working with Athar Munir Siddiqui on this study allowed me to apply my background in language testing and rater behaviour to a high-stakes context very different from the Spanish one.
Dr. Miguel Fernández Álvarez
Universidad Politecnica de Madrid
Read the Original
This page is a summary of: Markers’ criteria in assessing English essays: an exploratory study of the higher secondary school certificate (HSCC) in the Punjab province of Pakistan, Language Testing in Asia, March 2017, Springer Science + Business Media,
DOI: 10.1186/s40468-017-0037-0.
You can read the full text:
Resources
Contributors
The following have contributed to this page
