What is it about?
This paper reports on the results from the second GENEA Challenge, intended as a benchmark of data-driven automatic co-speech gesture generation. Participating teams used a common speech and motion dataset in the challenge to build gesture-generation systems. Motion generated by all these systems was then rendered to video using a standardised visualisation and evaluated in several large, crowdsourced user studies. This year’s dataset was based on 18 hours of full-body motion capture, including fingers, of different persons engaging in dyadic conversation, taken from the Talking With Hands 16.2M dataset. Ten teams participated in the evaluation across two tiers: full-body and upper-body gesticulation. For each tier, we evaluated both the human-likeness of the gesture motion and its appropriateness for the specific speech.
Featured Image
Photo by Callum Pritchard on Unsplash
Why is it important?
In general, challenges like the one described here can play an important part in identifying key factors for generating convincing co-speech gestures in practice, and help drive and validate future progress toward the goal of endowing embodied agents with natural and appropriate gesture motion.
Perspectives
The evaluation results are a revolution, and a revelation: Some synthetic conditions are rated as significantly more human-like than human motion capture. At the same time, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings.
Taras Kucherenko
Read the Original
This page is a summary of: The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation, November 2022, ACM (Association for Computing Machinery),
DOI: 10.1145/3536221.3558058.
You can read the full text:
Resources
Contributors
The following have contributed to this page
