What is it about?
This paper details the design and implementation of an immersive multimodal mixed reality system created to revitalize and preserve Nüshu culture, the unique script historically used exclusively by women in China and recognized as a National Intangible Cultural Heritage. Centered on the cultural artifact The Song of Nüshu, the system integrates Natural Interaction and Mixed Reality technologies to construct a visual and auditory environment inspired by classical Chinese landscape aesthetics. Users engage in a participatory storytelling experience through physical interactions, such as causing Nüshu characters to illuminate for modern translation (symbolizing cultural awakening) or to disintegrate like sand (symbolizing historical decline and female empowerment). The system’s modular architecture utilizes sensors like Kinect and Leap Motion to track user presence and gestures, feeding the data to an interactive control system that renders and delivers the digital content through projection and audio outputs. Ultimately, this work advances an innovative heritage preservation methodology that explicitly foregrounds women's historical agency and promotes the continuous evolution of this intangible culture.
Featured Image
Why is it important?
The system described in the paper is important because it offers a significant new methodology for intangible cultural heritage preservation. By integrating Multimodal Mixed Reality (MR) and Natural Interaction (NI) technologies, the project innovatively reinterprets Nüshu, the women's exclusive script, transforming its experience from passive consumption to participatory co-creation. This approach directly addresses the urgent need to revitalize Nüshu as a living heritage , allowing traditional female expressions to be revived within contemporary forms of expression. More broadly, the work is vital for overcoming physical and temporal barriers inherent in heritage preservation , and it uniquely highlights the proactive role of digital technologies in foregrounding women's historical agency and enabling marginalized voices to reemerge in contemporary contexts. The combination of visual, auditory metaphors, and user behavior stimulates emotional resonance and cultural reflection , making the heritage relevant and accessible to modern audiences, particularly youth.
Perspectives
The system and methodology presented in this paper offer several important perspectives on the future of cultural heritage preservation and digital media. Firstly, it underscores the potential of multimodal mixed reality systems in the inheritance and reinterpretation of intangible cultural heritage like Nüshu. By weaving together text, sound, and interactive behaviors, these systems move beyond simple documentation to create immersive and participatory cultural spaces where traditional female expressions are actively revived in contemporary forms. Secondly, the project advocates for a new approach to heritage work that explicitly foregrounds marginalized voices and women's historical agency, using digital tools to ensure these narratives reemerge in modern contexts. Finally, it provides a crucial perspective on the role of technology in fostering emotional resonance and cultural reflection , demonstrating that augmented, virtual, and mixed reality are pivotal in overcoming physical and temporal barriers, thereby promoting the continuous evolution of cultural traditions.
Zheyu Feng
Communication University of China
Read the Original
This page is a summary of: Reconstructing the Experience of Nüshu Culture: An Exploration via Multimodal Mixed Reality Systems, October 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3746027.3756145.
You can read the full text:
Contributors
The following have contributed to this page
