Can UAVs Understand Where They Are and How They Move?

What is it about?

Unmanned aerial vehicles (UAVs), commonly known as drones, are increasingly used in applications such as aerial inspection, environmental monitoring, and autonomous navigation. To operate intelligently, a UAV needs more than the ability to recognize objects and understand its surroundings. It must also understand its own movement, remember its previous actions, and reason about how its position and viewpoint change over time. In this work, we introduce SIS-Bench, a comprehensive benchmark designed to evaluate both spatial understanding and self-awareness in UAV-based artificial intelligence systems. The benchmark contains 4,856 questions derived from 1,646 real-world UAV videos, covering 13 tasks across three cognitive levels: perception, memory, and reasoning. We evaluate 26 multimodal large language models and identify important limitations in their ability to understand their own motion and actions. Our experiments reveal that current models generally understand the surrounding environment better than their own state, and struggle increasingly as tasks require memory and deeper reasoning. To address these limitations, we further introduce SIS-Motion, a motion-aware approach that combines visual information with explicit motion cues. Our results demonstrate that incorporating motion information improves both spatial understanding and self-awareness, with further benefits for downstream UAV navigation. Here, self-awareness refers to understanding the UAV's own movement and action state, rather than human-like consciousness.

Featured Image

Photo by Ian An on Unsplash

Photo by Ian An on Unsplash

Why is it important?

For autonomous UAVs, understanding the environment is only part of the problem. A UAV is both an observer and an active agent moving through the world. Its viewpoint continuously changes as it flies, and reliable decision-making requires understanding not only what is happening around it, but also how its own movements influence what it observes. However, existing AI systems and benchmarks have largely focused on recognizing external scenes, leaving the understanding of an agent's own state relatively unexplored. Our work addresses this gap by providing a systematic way to evaluate both spatial cognition and self-awareness in embodied UAV intelligence. Our evaluation shows that even the best-performing model achieves 71.6% overall accuracy, compared with 91.7% for human participants, highlighting substantial room for improvement. By demonstrating the value of explicit motion information, this research provides a practical direction for developing more capable AI systems that can better understand their surroundings, track their own movements, and make informed decisions in dynamic real-world environments.

Perspectives