What is it about?
Large-scale responsive environments are open spaces where the user’s presence, motion and gestures can be tracked by a computer system equipped with cameras or/and motion sensors. Computer vision algorithms process the incoming data by producing the coordinates of the users position or limb movements, which can be used to provide an audio or graphical output coherent with the user’s actions. This builds a strong relationship between the position – or movements – of the users and the environment around them. In recent years, applications based on this kind of technology have risen up as a convincing aid for learning simple as well as complex concepts in a playful way (https://www.wizefloor.com/). The full-body interaction that characterizes these environments supports different learning styles and it is particularly fit for inclusion of participants with disabilities, due to its richness in emotional engagement and ease of use.
Featured Image
Why is it important?
One possible use of this technology is for music didactics. An example is Harmonic Walk. an application devoted to the study of tonal harmony and melody harmonization. The users can play the different musical chords arranged on the application’s floor reaching them through a simple step (Mandanici et al. 2017). In this way it is possible to harmonize a tonal melody learning the movements necessary to reach the chords in due time with the melody’s harmonic changes. The large-scale responsive environment has also been used as a training tool designed to help blind children to avoid veering, employing jingle sounds, a natural soundscape, interactive sonification and full-body, hand-free interaction (Mandanici et al. 2018).
Perspectives
Assessment results of the various applications show that the great majority of participants were successful in the proposed tasks, whereas informal tests recorded interest, curiosity and great engagement, particularly from the children. Morevoer the application for blind children has proven to be an efficient tool to avoid veering and to encourage children to explore the surrounding space. These experimental data boost the employment of these responsive environments as real supportive tools for inclusive education, training and impaired children rehabilitation.
Marcella Mandanici
Music Conservatory Brescia
Read the Original
This page is a summary of: Inclusive sound and music serious games in a large-scale responsive environment, British Journal of Educational Technology, July 2018, Wiley,
DOI: 10.1111/bjet.12630.
You can read the full text:
Resources
Bodily Interactions in Motion-Based Music Applications
Motion-based music applications exploit the connection between body movements and musical concepts to allow users to practice high-level structured elements (e.g., tonal harmony) in a simple and effective way. We propose a framework for the design and the assessment of motion-based music applications by involving outcomes from various disciplines, such as the cognitive sciences and human–computer interaction. The framework has been applied to a working system, the Harmonic Walk, which is an interactive space application based on motion-tracking technologies. The application integrates both digital and physical information by reacting to a user’s movements within a designated 3 x 4 m floor, where six musical chords have been arranged according to a determined spatial positioning. Human choreographies from the user’s coordinated movements to musically structured events are analyzed in order to determine their relationships and to discuss related design issues.
Following the Cuckoo Sound: A Responsive Floor to Train Blind Children to Avoid Veering
Following the Cuckoo Sound is a responsive floor application designed to train blind children to walk along a straight path through the use of interactive audio. The system, based on computer vision algorithms, is able to estimate the cartesian coordinates of a user as soon as s/he enters the active area. To provide children with an efficient and playful tool which can help them in the task of walking straight, we designed an interactive audio environment based on a cuckoo sound which is altered if the child veers from a central hallway. To obtain preliminary information about the potential of the application, we organized a pilot study involving 6 blind children. Results show an improvement in the stability of the direction in 5 subjects out of 6. Moreover, the great interest among the children for interactive audio suggests that this is a promising investigation field to help wayfinding and orientation.
Contributors
The following have contributed to this page
