BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171530Z
LOCATION:Hall B5 (2)\, B Block\, Level 5
DTSTART;TZID=Asia/Tokyo:20241203T144500
DTEND;TZID=Asia/Tokyo:20241203T145600
UID:siggraphasia_SIGGRAPH Asia 2024_sess107_papers_701@linklings.com
SUMMARY:DifFRelight: Diffusion-Based Facial Performance Relighting
DESCRIPTION:Mingming He (Netflix Eyeline Studios); Pascal Clausen (Netflix
  Eyeline Studios, Osylum); and Ahmet Levent Taşel, Li Ma, Oliver Pilarski,
  Wenqi Xian, Laszlo Rikker, Xueming Yu, Ryan Burgert, Ning Yu, and Paul De
 bevec (Netflix Eyeline Studios)\n\nWe present a novel framework for free-v
 iewpoint facial performance relighting using diffusion-based image-to-imag
 e translation. Leveraging a subject-specific dataset containing diverse fa
 cial expressions captured under various lighting conditions, including fla
 t-lit and one-light-at-a-time (OLAT) scenarios, we train a diffusion model
  for precise lighting control, enabling high-fidelity relit facial images 
 from flat-lit inputs. Our framework includes spatially-aligned conditionin
 g of flat-lit captures and random noise, along with integrated lighting in
 formation for global control, utilizing prior knowledge from the pre-train
 ed Stable Diffusion model. This model is then applied to dynamic facial pe
 rformances captured in a consistent flat-lit environment and reconstructed
  for novel-view synthesis using a scalable dynamic 3D Gaussian Splatting m
 ethod to maintain quality and consistency in the relit results. In additio
 n, we introduce unified lighting control by integrating a novel area light
 ing representation with directional lighting, allowing for joint adjustmen
 ts in light size and direction. We also enable high dynamic range imaging 
 (HDRI) composition using multiple directional lights to produce dynamic se
 quences under complex lighting conditions. Our evaluations demonstrate the
  model's efficiency in achieving precise lighting control and generalizing
  across various facial expressions while preserving detailed features such
  as skin texture and hair. The model accurately reproduces complex lightin
 g effects like eye reflections, subsurface scattering, self-shadowing, and
  translucency, advancing photorealism within our framework.\n\nRegistratio
 n Category: Full Access, Full Access Supporter\n\nLanguage Format: English
  Language\n\nSession Chair: Hongzhi Wu (Zhejiang University; State Key Lab
 oratory of CAD&CG, Zhejiang University)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_701&sess=sess107
END:VEVENT
END:VCALENDAR