Understanding Freehand Cursorless Pointing Variability

Dataset for "Understanding Freehand Cursorless Pointing Variability and Its Impact on Selection Performance"

This dataset supports the journal entry "Understanding Freehand Cursorless Pointing Variability and Its Impact on Selection Performance" (TOCHI, 2025), containing motion capture data, of the body and hands, captured during a range of pointing gestures from 23 participants. The user study that captured this data systematically explored how target position (3 rows by 5 columns), task focus (Pointing as a Primary Task vs Secondary Task), and user effort (Accurate pointing vs Casual pointing), affect pointing behaviour and performance. The dataset includes: - Motion capture data for each trial (grouped by participant). This contains body landmarks – captured via a markerless motion capture system and finger landmarks – tracked with infrared markers. - Trial Annotations. Metadata for each trial, such as the target position, labels for when pointing occurs, and observed behaviour labels. - Encoded gesture statistics. For each trial, for which a valid pointing gesture could be extracted, an encoding of the gesture performed, derived from the medians for body pose features (e.g. elbow flexion), fatigue measures (e.g. consumed endurance), and rays (e.g. vector and accuracy). - Self-reported user data. Including participant age, hand dominance, and fatigue measures (obtained after completing pointing within each condition). - Code for visualising the trials, including a subset of the rays used in our subsequent analysis, code for generating our encoded gestures, using the motion capture data and annotations, and the script used to perform the analysis over our encoded gestures. This dataset has been provided for two purposes: 1. For further investigation into pointing behaviour and for the development of pointing interaction systems. For this, please refer to the Pointing Dataset section of the README to understand the structure and dataset contents, and the Trial Visualiser section of the README for usage of a script for visualising the motion capture data. 2. For reproduction of data used in the analysis of the accompanying paper (Understanding Freehand Cursorless Pointing Variability and Its Impact on Selection Performance), for which please see Pointing Dataset section of the README to understand the structure and dataset contents, along with the Gesture Encoder and Analysis Script sections of the README for the code used to perform our analysis.