BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20250110T023313Z
LOCATION:Hall B5 (2)\, B Block\, Level 5
DTSTART;TZID=Asia/Tokyo:20241206T104500
DTEND;TZID=Asia/Tokyo:20241206T105600
UID:siggraphasia_SIGGRAPH Asia 2024_sess143_papers_1185@linklings.com
SUMMARY:Gaussian Surfel Splatting for Live Human Performance Capture
DESCRIPTION:Technical Papers\n\nZheng Dong (State Key Laboratory of CAD&CG
 , Zhejiang University); Ke Xu (City University of Hong Kong); Yaoan Gao, H
 ujun Bao, and Weiwei Xu (State Key Laboratory of CAD&CG, Zhejiang Universi
 ty); and Rynson W.H. Lau (City University of Hong Kong)\n\nHigh-quality re
 al-time rendering using user-affordable capture rigs is an essential prope
 rty of human performance capture systems for real-world applications. Howe
 ver, state-of-the-art performance capture methods may not yield satisfacto
 ry rendering results under a very sparse (e.g., four) capture setting. Spe
 cifically, neural radiance field (NeRF)-based methods and 3D Gaussian Spla
 tting (3DGS)-based methods tend to produce local geometry errors for unsee
 n performers, while occupancy field (PIFu)-based methods often produce unr
 ealistic rendering results. In this paper, we propose a novel generalizabl
 e neural approach to reconstruct and render the performers from very spars
 e RGBD streams in high quality. The core of our method is a novel point-ba
 sed generalizable human (PGH) representation conditioned on the pixel-alig
 ned RGBD features. The PGH representation learns a surface implicit functi
 on for the regression of surface points and a Gaussian implicit function f
 or parameterizing the radiance fields of the regressed surface points with
  2D Gaussian surfels, and uses surfel splatting for fast rendering. We lea
 rn this hybrid human representation via two novel networks. First, we prop
 ose a novel point-regressing network (PRNet) with a depth-guided point clo
 ud initialization (DPI) method to regress an accurate surface point cloud 
 based on the denoised depth information. Second, we propose a novel neural
  blending-based surfel splatting network (SPNet) to render high-quality ge
 ometries and appearances in novel views based on the regressed surface poi
 nts and high-resolution RGBD features of adjacent views. Our method produc
 es free-view human performance videos of 1K resolution at 12 fps on averag
 e. Experiments on two benchmarks show that our method outperforms state-of
 -the-art human performance capture methods.\n\nRegistration Category: Full
  Access, Full Access Supporter\n\nLanguage Format: English Language\n\nSes
 sion Chair: Iain Matthews (Epic Games, Carnegie Mellon University)
URL:https://asia.siggraph.org/2024/program/?id=papers_1185&sess=sess143
END:VEVENT
END:VCALENDAR