What is it about?
Smartphone applications that allow users to enjoy playing musical instruments have emerged, opening up numerous related opportunities. However, it is difficult for deaf and hard of hearing (DHH) people to use these apps because of limited access to auditory information. When using real instruments, DHH people typically feel the music from the vibrations transmitted by the instruments or the movements of the body, which is not possible when playing with these apps. We introduce “smartphone drum,” a smartphone application that presents a drum-like vibrotactile sensation when the user makes a drumming motion in the air with their smartphone like a drumstick. We implemented an early prototype and received feedback from six DHH participants. We discuss the technical implementation and the future of new instruments of vibration.
Featured Image
Photo by Denise Jans on Unsplash
Why is it important?
Prior research has aided DHH people in playing musical instruments through sensory substitution. However, they assume that the instruments are already at hand and require specialized hardware, leaving a significant distance between DHH people and handy, everyday music playing. Considering this, we aimed to create a musical instrument that does not rely on sound but can be easily played by DHH people. To achieve this, we introduced the ``smartphone drum,'' a smartphone application that presents a drum-like vibrotactile sensation when the user makes a drumming motion in the air with their smartphone like a drumstick.
Perspectives
We designed the vibration patterns empirically, which may have caused discrepancies between a smartphone drum and acoustic drums, specifically for those who have heard the sound of acoustic drums. Exploring the design space of vibration patterns that are comfortable for users will be the subject of a future study. The primary objective behind this research was to assess the acceptance of the concept of a vibration-based musical instrument smartphone-app by DHH users and to understand their needs and capabilities. Therefore, it remains as a future task to investigate the effects of the nature and delay of vibrations on user experience and sensorimotor synchronization. Based on feedback from users, we would like to consider creating instruments other than drums, as well as creating a system for multiple users to play together.
Ryo Iijima
Tsukuba Daigaku
Read the Original
This page is a summary of: Smartphone Drum: Gesture-based Digital Musical Instruments Application for Deaf and Hard of Hearing People, November 2021, ACM (Association for Computing Machinery),
DOI: 10.1145/3485279.3488285.
You can read the full text:
Resources
Contributors
The following have contributed to this page
