The Lips, the Teeth, the tip of the Tongue: LTT Tracking
DescriptionA mesh-based generative model of the inner-mouth system is presented, which includes teeth and gums for the upper and lower jaw, the tongue, and their placement inside the human head. The model is capable of capturing person-specific detail, enabling the creation of highly accurate avatars that exceed the quality of prior mesh-based representations. The method combines data from dental mouth scans and facial performances captured in a multi-camera capture rig. The system employs a precise segmentation model that can differentiate complex tongue motion. A novel inverse-rendering formulation is used in a staged modeling procedure, producing accurate registration of tongue, teeth, and jaw as well as improved disentanglement of non-rigid face motion from rigid head motion. The system is demonstrated on novel held-out subjects, where we demonstrate highly accurate reconstructions that exceed prior mesh-based avatar representations.
Event Type
Technical Papers
TimeTuesday, 3 December 20249:00am - 12:00pm JST
LocationHall C, C Block, Level 4
