BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171532Z
LOCATION:Hall B5 (2)\, B Block\, Level 5
DTSTART;TZID=Asia/Tokyo:20241206T130000
DTEND;TZID=Asia/Tokyo:20241206T131100
UID:siggraphasia_SIGGRAPH Asia 2024_sess146_papers_359@linklings.com
SUMMARY:Correlation-aware Encoder-Decoder with Adapters for SVBRDF Acquisi
 tion
DESCRIPTION:Di Luo and Hanxiao Sun (Nankai University), Lei Ma (Peking Uni
 versity), Jian Yang (Nankai University), and Beibei Wang (Nanjing Universi
 ty)\n\nCapturing materials from the real world avoids laborious manual mat
 erial authoring. However, recovering high-fidelity Spatially Varying Bidir
 ectional Reflectance Distribution Function (SVBRDF) maps from a few captur
 ed images is challenging due to its ill-posed nature. Existing approaches 
 have made extensive efforts to alleviate this ambiguity issue by leveragin
 g generative models with latent space optimization or extracting features 
 with variant encoder-decoders. Albeit the rendered images at input views c
 an match input images, the problematic decomposition among maps leads to s
 ignificant differences when rendered under novel views/lighting. We observ
 e that for human eyes, besides individual images, the correlation (or the 
 highlights variation) among input images also serves as an important hint 
 to recognize the materials of objects. Hence, our key insight is to explic
 itly model this correlation in the SVBRDF acquisition network. To this end
 , we propose a correlation-aware encoder-decoder network to model the corr
 elation features among the input images via a graph convolutional network 
 by treating channel features from each image as a graph node. This way, th
 e ambiguity among the maps has been reduced significantly. However, severa
 l SVBRDF maps still tend to be over-smooth, leading to a mismatch in the n
 ovel-view rendering. The main reason is the uneven update of different map
 s caused by a single decoder for map interpretation. To address this issue
 , we further design an adapter-equipped decoder consisting of a main decod
 er and four tiny per-map adapters, where adapters are employed for individ
 ual maps interpretation, together with fine-tuning, to enhance flexibility
 . As a result, our framework allows the optimization of the latent space w
 ith the input image feature embeddings as the initial latent vector and th
 e fine-tuning of per-map adapters. Consequently, our method can outperform
  existing approaches both visually and quantitatively on synthetic and rea
 l data.\n\nRegistration Category: Full Access, Full Access Supporter\n\nLa
 nguage Format: English Language\n\nSession Chair: Valentin Deschaintre (Ad
 obe Research)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_359&sess=sess146
END:VEVENT
END:VCALENDAR