What is it about?
Voice assistants like Siri or Alexa listen for a specific 'wake word' (like "Hey Siri") before they activate. We've developed a new, smarter AI system to detect these wake words more accurately and efficiently, especially when the audio is streaming live. Our system, called 3W-ResSC, is inspired by how humans learn from multiple perspectives. It combines three key AI techniques: - Residual Learning (ResNet): Helps the AI learn deeply without losing important details. - CNN Mixer (Mi): Allows the AI to look at sound patterns from different angles (point-wise and depth-wise). - Attention (At): A special mechanism we designed called 'independent Multi-View Attention' (iMVA) helps the AI focus on the most important parts of the sound to make a decision, even in a continuous stream of audio. We also created a complete framework that processes speech in small, independent chunks, mimicking how devices handle live audio.
Featured Image
Photo by omid armin on Unsplash
Why is it important?
For voice assistants on your phone or smart speaker to work well, they need to quickly and reliably hear their wake word without using too much power or memory. Our research is important because: - Better Performance, Less Power: Our 3W-ResSC system is more accurate at detecting wake words than many existing methods, as shown by lower error rates on several datasets. Importantly, it's designed to be lightweight, meeting the requirements for devices with limited processing power. For example, one of our best combinations (ResNet+Mi) uses only about 56,000 parameters and 8 million computation units (MACs). - Smart Combinations for Different Needs: We found that different combinations of our three core techniques (ResNet, Mixer, Attention) work best depending on the situation. For example, 'ResNet+Mixer' is great for small amounts of training data or when distinguishing between many keywords, while 'ResNet+Attention' excels with large datasets. This flexibility makes our approach adaptable. - Designed for Real-World Streaming: The entire system, including our novel 'independent Multi-View Attention,' is built to handle live, streaming audio effectively. This is crucial for how these devices operate in everyday use, as our attention mechanism (iMVA) doesn't need to wait for future audio to make a decision.
Perspectives
Developing the 3W-ResSC model and the streaming framework was a really exciting challenge. The idea of mimicking human multi-perspective learning – how we can look at a problem from different angles and combine that understanding – was a core inspiration for our model's design. We wanted to translate this into an AI model that could efficiently 'listen' for wake words. It was fascinating to see how combining established concepts like ResNet with newer ideas like CNN Mixers and our own adaptation of attention (iMVA) could lead to significant performance gains. What I find particularly promising is not just the improved accuracy, but also the adaptability of the different combinations (ResNet+Mi, ResNet+At, ResNet+Mi+At) to various scenarios, like different amounts of training data or types of wake word tasks. This means the approach can be tailored for specific needs, which is vital for real-world applications on diverse devices. Our aim was to create something that's not only effective but also practical for the demands of streaming audio processing, and I believe this work takes a solid step in that direction.
Sattaya Singkul
True Digital Group
Read the Original
This page is a summary of: Residual, Mixer, and Attention: The Three-way Combination for Streaming Wake Word Detection Framework, October 2023, Institute of Electrical & Electronics Engineers (IEEE),
DOI: 10.1109/apsipaasc58517.2023.10317514.
You can read the full text:
Contributors
The following have contributed to this page
