Understanding how AI training affects human behavior

What is it about?

From recommending new songs and shows to helping us find our way around a city, artificial intelligence (AI) has seamlessly integrated into our daily lives. Beyond these everyday tasks, AI is increasingly used to help with crucial decisions, such as assisting doctors with patient care and judges with sentencing. Typically, AI is trained by observing human behavior, assuming that people act according to their default preferences. However, across five experiments, we show that this is not the case. Specifically, when participants knew their choices would train AI, they changed their behavior to instill desired behaviors into the AI algorithm. Strikingly, when these participants returned a few days later to complete the same task without AI training, they continued to behave as if they were still training AI. This suggests that their modified behavior had become habitual. Our research shows that using human behavior to train AI has more consequences than previously thought since it (a) can produce AI to perpetuate people’s biases when helping people make choices and (b) cause people to form habits that deviate from how they would normally act.

Featured Image

Photo by Steve Johnson on Unsplash

Photo by Steve Johnson on Unsplash

Why is it important?

Our research team was the first to demonstrate that people instill desired behaviors in AI algorithms that do not align with their normal actions when given the chance to teach AI how to make choices. This behavior shift will lead AI to perpetuate people’s biases when helping people make decisions. We also found that this modified behavior persisted after AI training, becoming a lasting habit. Our research highlights an issue with current practice by invalidating the assumption that people will act according to their default preferences when training AI. Therefore, this research underscores the need for AI developers to account for these behavioral shifts to ensure that AI systems produce unbiased and accurate recommendations. Additionally, our work enhances cognitive science by deepening our understanding of how people integrate AI into their decision-making processes and how AI use affects behavior.

Perspectives