What is it about?
Educational testing programs are using AI-based, automated scoring tools for essay programs and some short response items. These programs are required to support the validity of interpretations based on these scores using claims, evidence, and validity arguments. These validity arguments can be a challenge for AI-based scores that are only partly explainable (e.g., word count vs. n-grams) but that are necessary for scoring accuracy.
Featured Image
Why is it important?
As programs expand their use of AI-based automated scoring, they will need to provide evidence and develop arguments to support valid interpretations of scores that are generated from partially unexplainable, AI-based scoring tools.
Perspectives
We have to make room for partially unexplainable AI-based scoring tools where unexplainable variables are necessary for scoring accuracy. The goal of these models is not to mimic how humans make scoring decisions. Rather, the goal is to build scoring tools that produce scores that are as accurate or more accurate than human scores.
Steve Ferrara
Cognia Alpharetta GA USA
Read the Original
This page is a summary of: Validity Arguments for AI‐Based Automated Scores: Essay Scoring as an Illustration, Journal of Educational Measurement, June 2022, Wiley,
DOI: 10.1111/jedm.12333.
You can read the full text:
Contributors
The following have contributed to this page
