From Skin to Skeleton : Towards Biomechanically Accurate 3D Digital Humans
SessionFlesh & Bones
DescriptionGreat progress has been made in estimating 3D human pose and shape from images and video by training neural networks to directly regress the parameters of parametric human models like SMPL.
However, existing body models have simplified kinematic structures that do not correspond to accurate joint locations and articulations in the human skeletal system, limiting their potential use in biomechanics. On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods.
What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily regressed from images.
To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the true joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters.
The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom. We also train a regressor from SMPL meshes to the skeleton enabling us to ``upgrade" existing datasets that are in SMPL format. We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods.
SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation.
However, existing body models have simplified kinematic structures that do not correspond to accurate joint locations and articulations in the human skeletal system, limiting their potential use in biomechanics. On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods.
What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily regressed from images.
To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the true joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters.
The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom. We also train a regressor from SMPL meshes to the skeleton enabling us to ``upgrade" existing datasets that are in SMPL format. We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods.
SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation.
Event Type
Technical Communications
Technical Papers
TimeThursday, 14 December 20234:35pm - 4:50pm 
LocationMeeting Room C4.8, Level 4 (Convention Centre)
