Creating digital human and digital reconstruction.

What is it about?

This paper focuses on human pose estimation, subject-specific modeling, texture reconstruction, which is basically related to digital human creation. 3-D human pose estimation or human tracking hasalways been the focus of research in the human–computer interaction community. As the calibration step of human pose estimation, subject-specific modeling is crucially important to the subsequent pose estimation process. It not only provides a priori knowledge but also clearly defines the tracking target. This article presents a fully automatic subject modeling framework to reconstruct human pose, shape, as well as the body texture in a challenging optimization scenario. By integrating powerful differentiable rendering into the subject-specific modeling pipeline, the proposed method transforms the texture reconstruction problem into analysis by synthesis minimization and solves it efficiently by a gradient-based method. Furthermore, a novel covariance matrix adaptation annealing algorithm is proposed to attack the high-dimensional multimodal optimization problem in an adaptive manner. The domain knowledge of hierarchical human anatomy is seamlessly injected to the annealing optimization process by using a soft covariance matrix mask. All together contributes to the novel algorithm robust to the temptation of local minima. Experiments on the Human3.6 M dataset and the People-Snapshot dataset demonstrate the competitive results to the state of the art both qualitatively and quantitatively.

Featured Image

Photo by Pawel Czerwinski on Unsplash

Photo by Pawel Czerwinski on Unsplash

Why is it important?

Our world is becoming more digital and virtual. This paper addresses the future technology of reconstructing 3D digital human.

Perspectives