A Cognitive Scaling Mixer for Concurrent Ultrasound Sensing and Music Playback in Smart Devices

What is it about?

Recent advances in the field of acoustic sensing have enabled various applications in different domains including mobile health (biosignals monitoring), human-computer interaction (gesture recognition), and imaging. These acoustic sensing systems typically leverage the existing speakers and microphones in COTS smart devices by transforming them into a SONAR system that can detect minute motions in the environment. Although this is beneficial, the sensing systems might negatively affect the traditional utilities of these sensors, which is to play/record music and voice. For example, transmitting ultrasonic sound signals concurrently with music on a smart speaker can lead to overload in the speaker’s mixer, resulting in a degraded quality of both music and sensing. In this paper, we address this problem by cognitively adapting the sensing systems, so that they can work in concurrence with the music playback without any degradation in the music quality that can be interpreted by the users and also achieving optimal sensing accuracy. We enable this by formalizing it as an optimization problem that maximizes the transmitted sensing signal magnitude and minimizes the distortion conditioned on the concurrent music play. Specifically, we design a deep learning model that takes a high-frequency sine wave sensing signal and generates an adapted sensing signal that ensures both the accuracy of sensing and the quality of music play simultaneously. We conduct a small pilot study to validate our approach in a downstream task of respiration monitoring. The results show that the adapted signal achieves similar accuracy in respiration signal detection with scenarios with no concurrent music.

Featured Image

Photo by Lazar Gugleta on Unsplash

Photo by Lazar Gugleta on Unsplash

Why is it important?

Acoustic sensing has trouble co-existing with other music playback from the same speaker. This results in a degraded quality of both music and sensing. So, we propose a learning-based method to cognitively scale the sensing signal to avoid this problem.

Perspectives