BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171530Z
LOCATION:Hall B7 (1)\, B Block\, Level 7
DTSTART;TZID=Asia/Tokyo:20241205T171600
DTEND;TZID=Asia/Tokyo:20241205T172800
UID:siggraphasia_SIGGRAPH Asia 2024_sess138_papers_298@linklings.com
SUMMARY:Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Port
 rait Animation
DESCRIPTION:Yue Ma and Hongyu Liu (Hong Kong University of Science and Tec
 hnology); Hongfa Wang and Heng Pan (Tencent); Yingqing He (Hong Kong Unive
 rsity of Science and Technology); Junkun Yuan, Ailing Zeng, and Chengfei C
 ai (Tencent); Heung-Yeung Shum (Tsinghua University); Wei Liu (Tencent); a
 nd Qifeng Chen (Hong Kong University of Science and Technology)\n\nWe pres
 ent Follow-Your-Emoji, a diffusion-based framework for portrait animation,
  which animates a reference portrait with target landmark sequences. The m
 ain challenge of portrait animation is to preserve the identity of the ref
 erence portrait and transfer the target expression to this portrait while 
 maintaining temporal consistency and fidelity. To address these challenges
 , Follow-Your-Emoji equipped the powerful Stable Diffusion model with two 
 well-designed technologies. Specifically,  we first adopt a new explicit m
 otion signal, namely expression-aware landmark,  to guide the animation pr
 ocess. We discover this landmark can not only ensure the accurate motion a
 lignment between the reference portrait and target motion during inference
  but also increase the ability to portray exaggerated expressions (i.e., l
 arge pupil movements) and avoid identity leakage. Then, we propose a facia
 l fine-grained loss to improve the model's ability of subtle expression pe
 rception and reference portrait appearance reconstruction by using both ex
 pression and facial masks. Accordingly, our method demonstrates significan
 t performance in controlling the expression of freestyle portraits, includ
 ing real humans, cartoons, sculptures, and even animals. By leveraging a s
 imple and effective progressive generation strategy, we extend our model t
 o stable long-term animation, thus increasing its potential application va
 lue. To address the lack of a benchmark for this field, we introduce Emoji
 Bench, a comprehensive benchmark comprising diverse portrait images, drivi
 ng videos, and landmarks. We show extensive evaluations on EmojiBench to v
 erify the superiority of Follow-Your-Emoji.\n\nRegistration Category: Full
  Access, Full Access Supporter\n\nLanguage Format: English Language\n\nSes
 sion Chair: Hongbo Fu (Hong Kong University of Science and Technology)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_298&sess=sess138
END:VEVENT
END:VCALENDAR