Curiosity-driven learning of vision-based robotic reaching and grasping

What is it about?

Learning motor behavior purely from experience and without any prior knowledge of the task or the underlying physics is a challenging problem in robot learning. It becomes more difficult in realistic robotic environments where noisy input, sparse feedback, and real-time constraints are common. We address this problem by first learning a simple feature representation that encodes information sufficient to identify rewarding states and reconstruct the original observation. We then learn in an unsupervised fashion local predictive models that take in the current encoded state and action and predict the future encoded state. We show that the learning progress of this ensemble of dynamics models provides a rich intrinsic reward enabling more directed exploration for our humanoid robot that learns grasping skills using reinforcement learning from sparse feedback and raw visual input. The results show state-of-the-art performance of our approach for learning-to-reach and learning-to-grasp tasks in different reward settings.

Featured Image

Why is it important?

Our work addresses a number of key challenges of utilizing deep neural networks in robotics, particularly the data efficiency of deep learning-based approaches for continuous motor skill acquisition and learning in environments with sparse reward signals.

Perspectives