BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Australia/Melbourne
X-LIC-LOCATION:Australia/Melbourne
BEGIN:DAYLIGHT
TZOFFSETFROM:+1000
TZOFFSETTO:+1100
TZNAME:AEDT
DTSTART:19721003T020000
RRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=1SU
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:19721003T020000
TZOFFSETFROM:+1100
TZOFFSETTO:+1000
TZNAME:AEST
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20240214T070248Z
LOCATION:Meeting Room C4.8\, Level 4 (Convention Centre)
DTSTART;TZID=Australia/Melbourne:20231214T154500
DTEND;TZID=Australia/Melbourne:20231214T155500
UID:siggraphasia_SIGGRAPH Asia 2023_sess158_papers_239@linklings.com
SUMMARY:Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views
DESCRIPTION:Technical Papers\n\nTaeho Kang and Kyungjin Lee (Seoul Nationa
 l University), Jinrui Zhang (Central South University), and Youngki Lee (S
 eoul National University)\n\nWe present Ego3DPose, a highly accurate binoc
 ular egocentric 3D pose reconstruction system. The binocular egocentric se
 tup offers practicality and usefulness in various applications, however, i
 t remains largely under-explored. It has been suffering from low pose esti
 mation accuracy due to viewing distortion, severe self-occlusion, and limi
 ted field-of-view of the joints in egocentric 2D images. Here, we notice t
 hat two important 3D cues, stereo correspondences, and perspective, contai
 ned in the egocentric binocular input are neglected. Current methods heavi
 ly rely on 2D image features, implicitly learning 3D information, which in
 troduces biases towards commonly observed motions and leads to low overall
  accuracy. We observe that they not only fail in challenging occlusion cas
 es but also in estimating visible joint positions. To address these challe
 nges, we propose two novel approaches. First, we design a two-path network
  architecture with a new path that estimates pose per limb independently. 
 It learns to output the 3D orientation of each limb with confidence based 
 on the specific limb’s heatmaps from a stereo view. It does not rely on fu
 ll-body information and alleviates bias toward learned full-body poses. Se
 cond, we leverage the egocentric view of body limbs, which exhibits strong
  perspective variance (e.g., a significantly large-size hand when it is cl
 ose to the camera). We propose a new perspective-aware representation usin
 g trigonometry, enabling the network to estimate the 3D orientation of lim
 bs. Finally, we develop an end-to-end pose reconstruction network that syn
 ergizes both techniques. Our comprehensive evaluations demonstrate that Eg
 o3DPose outperforms state-of-the-art models by a pose estimation error (i.
 e., MPJPE) reduction of 23.1% in the UnrealEgo dataset. Our qualitative re
 sults highlight the superiority of our approach across a range of scenario
 s and challenges\n\nRegistration Category: Full Access\n\nSession Chair: J
 ae-Ho Nah (Sangmyung University)
URL:https://asia.siggraph.org/2023/full-program?id=papers_239&sess=sess158
END:VEVENT
END:VCALENDAR