Empirical Research in Mid-Air Interaction: A Systematic Review

What is it about?

Mid-air interaction is a distinct style of natural HCI (Human-Computer Interaction). In mid-air interaction, users make use of their whole body—with a strong focus on hands—and apply gestures, postures, and movements to interact with digital content on distant displays or remote devices. Although the idea of exploiting body movements and gestures in HCI is not new, mid-air interaction is now possible due to the availability of reasonably priced depth cameras and sensors. This article presents a systematic review of empirical research in mid-air interaction, based on a corpus of 104 publications from 2011 to 2018. The review includes: (a) a retrospective on mid-air interaction, which outlines its historical development and clarifications on important concepts; (b) current application domains of mid-air interaction; (c) user requirements methods, focusing on gesture elicitation studies; (d) dimensions of prototyping and design; (e) empirical evaluation methods and issues; (f) a discussion on several trends and challenges for further research and development.

Featured Image

Why is it important?

Mid-air interaction is a fast-changing area of research and development, with many parallel developments in academia and the industry. The development of mid-air interaction applications is research- and design-intensive task that includes a lot of prototyping, iterative design and evaluation in a ‘research-through design’ approach. This survey presents the current landscape on empirical and design research in mid-air interaction. The emphasis of this survey is on empirical methods and approaches that provide user-defined proposals for mid-air gestures and/or evaluate systems on the basis of usability and the user experience. We have identified several trends (and challenges) for future development of mid-air interaction, namely: (i) exploration of mid-air interaction in various pervasive contexts of use; (ii) moving from user-defined gesture identification to automatic compilation of gesture properties; (iii) metaphoric mid-air interaction design; (iv) body-centered and on-skin mid-air gestures; (v) multimodal, natural interactions that combine mid-air gestures with other input modes; (vi) feedback mechanisms for mid-air interactions; (vii) data transfer and sharing among devices with midair-interactions; (viii) accessibility. We envisage that this survey will be informative and inspiring to HCI researchers and it may assist them to purposefully direct their study and research.