Building a Dance Castle, Block by Block

What is it about?

Dance is an important art form in human culture, but creating new dances can be both challenging and time-consuming. In this pa per, we propose a novel dance choreography framework, EDMG, designed to efficiently generate creative and long-lasting dance se quences conditioning on music and dance descriptions. In the first stage, we propose a flexible dance diffusion method, combined with dance genre description and descriptions of fundamental move ments to generate the dance sequences. To achieve high computa tional efficiency and inference speed, EDMG designs a lightweight denoising module by using selective parallel scanning algorithm from Mamba2. This Parallel Mamba Denoiser reduces significantly the number of parameters and accelerates remarkably both the learning and inference processes. In the second stage, by designing a smoothing module with a long receptive field, we mitigate joint error accumulation that causes jittering movements andfootsliding, thereby enhancing the fluency and visual appeal of the dance move ments. Furthermore, we extend the AIST++ dataset by adding de tailed descriptions of dance genres and fundamental movements, us ing the LargeLanguageModel(LLM).Thesedescriptionsfurtherim prove the choreography generation. EDMG is validated through ex tensive experiments, demonstrating that our method can both effec tively and efficiently generate long-term dances suitable for various dance genres.

Featured Image

Photo by Breakreate on Unsplash

Photo by Breakreate on Unsplash

Why is it important?

To achieve efficient long-term dance sequence generation, based on the diffusion model, we propose a new denoising module, the Parallel Mamba Denoiser. This module adopts the selective parallel scanning mechanism from Mamba2 and applies it to memorizing long dance sequences. To address the jittery outputs caused by directly applying Mamba2’s selective scanning mechanism to long-term dance generation, we design TRM. This module calculates 3D joint coordinates through fully connected layers with residual connections. These connections provide a wider temporal field of view. TRM optimizes the position, velocity, and acceleration of dance movements, which effectively improves the smoothness. To enhance dancers’ choreography freedom, we use LLM to generate text descriptions of dance genres as well as descriptions of fundamental movements for each dance genre. By utilizing these descriptions, we can generate more flexible fundamental dance movements.

Perspectives