BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171533Z
LOCATION:Hall B5 (2)\, B Block\, Level 5
DTSTART;TZID=Asia/Tokyo:20241204T134600
DTEND;TZID=Asia/Tokyo:20241204T135800
UID:siggraphasia_SIGGRAPH Asia 2024_sess116_papers_330@linklings.com
SUMMARY:Generative Portrait Shadow Removal
DESCRIPTION:Jae Shin Yoon, Zhixin Shu, Mengwei Ren, Xuaner Zhang, Yannick 
 Hold-Geoffroy, Krishna Kumar Singh, and He Zhang (Adobe Inc.)\n\nWe introd
 uce a high-fidelity portrait shadow removal model that can effectively enh
 ance the image of a portrait by predicting its appearance under disturbing
  shadows and highlights. Portrait shadow removal is a highly ill-posed pro
 blem where multiple plausible solutions can be found based on a single ima
 ge. For example, disentangling complex environmental lighting from origina
 l skin color is a non-trivial problem. While existing works have solved th
 is problem by predicting the appearance residuals that can propagate local
  shadow distribution, such methods are often incomplete and lead to unnatu
 ral predictions, especially for portraits with hard shadows. We overcome t
 he limitations of existing local propagation methods by formulating the re
 moval problem as a generation task where a diffusion model learns to globa
 lly rebuild the human appearance from scratch as a condition of an input p
 ortrait image. For robust and natural shadow removal, we propose to train 
 the diffusion model with a compositional repurposing framework: a pre-trai
 ned text-guided image generation model is first fine-tuned to harmonize th
 e lighting and color of the foreground with a background scene by using a 
 background harmonization dataset; and then the model is further fine-tuned
  to generate a shadow-free portrait image via a shadow-paired dataset. To 
 overcome the limitation of losing fine details in the latent diffusion mod
 el, we propose a guided-upsampling network to restore the original high-fr
 equency details (e.g., wrinkles and dots) from the input image. To enable 
 our compositional training framework, we construct a high-fidelity and lar
 ge-scale dataset using a lightstage capturing system and synthetic graphic
 s simulation. Our generative framework effectively removes shadows caused 
 by both self and external occlusions while maintaining original lighting d
 istribution and high-frequency details. Our method also demonstrates robus
 tness to diverse subjects captured in real environments.\n\nRegistration C
 ategory: Full Access, Full Access Supporter\n\nLanguage Format: English La
 nguage\n\nSession Chair: Dani Lischinski (Hebrew University of Jerusalem, 
 Google)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_330&sess=sess116
END:VEVENT
END:VCALENDAR