When Do People Want an Explanation from a Robot?

What is it about?

Imagine a robot assisting in a kitchen but failing to grasp a coffee mug, dropping it on the floor. Or a robotic helper telling you it is unsure how to find the way to the living room. In moments like these, our research reveals, people are curious about what went wrong and want the robot to explain itself, offering a glimpse into its decision-making process. We comprehensively show in what contexts people want robots to be able to explain their actions. Our results are based on a literature review and verified empirically with a large group of participants who were given a variety of videos depicting interactions between a robot and people. Some of the situations in which participants were most interested in explanations were when the robot made an error, was unable to accomplish a task, or had to adapt its plan when encountering something unexpected. In many of the scenarios, we found not only that people want an explanation but also that they value explanations more than other types of communication, such as apologizing or simply saying what happened. Lastly, we also explore individual differences based on experience in robotics and attitudes towards robots, with the need for explanations being present across groups.

Featured Image

Photo by Alex Knight on Unsplash

Photo by Alex Knight on Unsplash

Why is it important?

Explanations are a burgeoning topic in AI and robotics. The need for explainable AI is caused by the increased use of hard-to-understand black box architectures such as deep neural networks, coupled with the increased real-world usage of developed systems. Explanations can help users understand if a system works correctly and reliably and help with coordination in collaborative tasks. However, a crucial aspect missing in much of explainable AI and robotics research is the consideration of the user who has to interact with the system. Therefore, we set out to identify the human–robot interaction contexts in which people want an explanation from the robot, thus allowing further research to be grounded in real user needs and preferences.