Understanding exploration vs exploitation during reward-based motor learning

What is it about?

This study looked at how people learn new movements by trial and error, specifically, when they choose to try new ways versus keep doing what already works. In the experiment, participants used a robotic arm to reach for a hidden target. When the movement was successful, they received a reward. For some people, the task became harder over time because the target got smaller. For others, the difficulty stayed the same. The researchers used machine learning methods to group similar movements together and track how people changed their movements from one attempt to the next. Here, exploration meant trying a different type of movement than before. Conversely, exploitation meant repeating a movement that had worked well. They found that when the task became harder (smaller targets), people experimented more and tried many different movements. However, too much experimenting actually slowed down learning, because people didn’t spend enough time repeating successful movements.

Featured Image

Photo by Rezli on Unsplash

Photo by Rezli on Unsplash

Why is it important?

This is a demonstration of employing an unsupervised machine learning technique to assess motor skill learning. Such trial-and-error learning is natural in real world settings, such as when one first plays golf or tennis. People learn best when they balance trying new strategies with repeating what works. Furthermore, making a task harder can push people to explore too much, which can hurt learning instead of helping it.