BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Australia/Melbourne
X-LIC-LOCATION:Australia/Melbourne
BEGIN:DAYLIGHT
TZOFFSETFROM:+1000
TZOFFSETTO:+1100
TZNAME:AEDT
DTSTART:19721003T020000
RRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=1SU
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:19721003T020000
TZOFFSETFROM:+1100
TZOFFSETTO:+1000
TZNAME:AEST
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260114T163643Z
LOCATION:Meeting Room C4.11\, Level 4 (Convention Centre)
DTSTART;TZID=Australia/Melbourne:20231215T155000
DTEND;TZID=Australia/Melbourne:20231215T160000
UID:siggraphasia_SIGGRAPH Asia 2023_sess139_papers_222@linklings.com
SUMMARY:Fusing Monocular Images and Sparse IMU Signals for Real-time Human
  Motion Capture
DESCRIPTION:Shaohua Pan, Qi Ma, and Xinyu Yi (Tsinghua University); Weifen
 g Hu, Xiong Wang, Xingkang ZHOU, and Jijunnan LI (OPPO Research Institute)
 ; and Feng Xu (Tsinghua University)\n\nEither RGB images or inertial signa
 ls have been used for the task of motion capture (mocap), but combining th
 em together is a new and interesting topic. We believe that the combinatio
 n is complementary and able to solve the inherent difficulties of using on
 e modality input, including occlusions, extreme lighting/texture, and out-
 of-view for visual mocap and global drifts for inertial mocap. To this end
 , we propose a method that fuses monocular images and sparse IMUs for real
 -time human motion capture. Our method contains a dual coordinate strategy
  to fully explore the IMU signals with different goals in motion capture. 
 To be specific, besides one branch transforming the IMU signals to the cam
 era coordinate system to combine with the image information, there is anot
 her branch to learn from the IMU signals in the body root coordinate syste
 m to better estimate body poses. Furthermore, a hidden state feedback mech
 anism is proposed for both two branches to compensate for their own drawba
 cks in extreme input cases. Thus our method can easily switch between the 
 two kinds of signals or combine them in different cases to achieve a robus
 t mocap. Quantitative and qualitative results demonstrate that by delicate
 ly designing the fusion method, our technique significantly outperforms th
 e state-of-the-art vision, IMU, and combined methods on both global orient
 ation and local pose estimation.\n\nRegistration Category: Full Access\n\n
 Session Chair: Yuting Ye (Reality Labs Research, Meta; Meta)\n\n
URL:https://asia.siggraph.org/2023/full-program?id=papers_222&sess=sess139
END:VEVENT
END:VCALENDAR