Creating 3D animated stories with GenAI

What is it about?

While 3D animation is widely used in commercial advertising and the movie industry, its application in academic settings has been underexplored, primarily due to the perceptions of high cost and complexity. This project aims to address the relevant barriers by designing instructional materials that enable students at various educational levels to narrate basic 3D animated stories using generative AI tools, motion libraries, and traditional 3D software. The authors developed a manageable and sustainable workflow encompassing AI-assisted storyboarding, 2D image creation, 3D character design, modelling and texturing, scene construction, motion integration, and audio mixing. It was subsequently implemented with 106 participants at a higher education institution, including 38 early childhood education pre-service teachers, 18 secondary education pre-service teachers specialising in STEAM, and 50 undergraduate students from AI and Education Technology disciplines. Over a 13-week semester, data were collected through pre- and post-questionnaires, as well as instructor research diaries that documented key classroom events and other defining moments, during which the 6-P (Plan, Prompt, Preview, Produce, Peer Review, Portfolio Tracking) pedagogical framework was introduced and evaluated. It was found that the model was generally feasible but with some limitations. More importantly, the integration of generative AI facilitated the implementation of two levels of motion capture technology in 3D animation creation. At Level One, the process involved: (1) using large language models (e.g., ChatGPT, Copilot) for story scriptwriting; (2) leveraging AI for 2D image generation to aid 3D modelling; (3) utilising royalty-free 3D characters; and (4) integrating 3D motion from existing libraries. At Level Two, the process included: (1) AI-assisted scriptwriting; (2) AI-generated 2D images for 3D modelling; (3) recording custom 3D motion using a mocap suit; and (4) integrating this motion with 3D models. The study concluded that this pedagogical innovation could not only enhance student motivation and engagement but also empower both students and instructors to visualise presentations and learned concepts more effectively, ultimately leveraging the combined power of AI and traditional software tools.

Featured Image

Photo by Umesh Soni on Unsplash

Photo by Umesh Soni on Unsplash