What is it about?
Motion and music are intertwined. We dance to the music; even when you play a classical musical instrument, you perform motion gestures. We combine the two to create a synesthesia experience where motion makes music and music affects movement. We achieve this using wireless WiFi sensors with an inertial measurement unit that sends OSC messages to a computer running Ableton DAW with a custom-made VST plugin. However, the system can also run standalone using the Processing Java framework or in any other DAW such as Cubase, Logic, or Reaper.
Featured Image
Photo by Yi Liu on Unsplash
Why is it important?
You can use musical software you already know. You do not need to code anything. Furthermore, the sensors are cheap and widely available. We also provide a template for Ableton and Processing Java so you can use it immediately. More importantly, you can also use the sensors for other use cases such as dance performances, tracking objects in VR, tracking sports performances, lighting control, and many others. We also simplify the usage of spatial audio for realtime performance.
Perspectives
I love juggling, and I enjoy music. But until now I couldn't play an instrument. I can transform my existing talent into other domains with the proposed system. I would love to see other people create new content with more freedom than before.
Vojtech Leischner
Czech Technical University in Prague
Read the Original
This page is a summary of: Sonification of a juggling performance using spatial audio, Proceedings of the ACM on Computer Graphics and Interactive Techniques, August 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3597619.
You can read the full text:
Resources
Contributors
The following have contributed to this page
