BEGIN:VCALENDAR VERSION:2.0 PRODID:Linklings LLC BEGIN:VTIMEZONE TZID:Asia/Tokyo X-LIC-LOCATION:Asia/Tokyo BEGIN:STANDARD TZOFFSETFROM:+0900 TZOFFSETTO:+0900 TZNAME:JST DTSTART:18871231T000000 END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTAMP:20250110T023312Z LOCATION:Hall B5 (2)\, B Block\, Level 5 DTSTART;TZID=Asia/Tokyo:20241205T163000 DTEND;TZID=Asia/Tokyo:20241205T164100 UID:siggraphasia_SIGGRAPH Asia 2024_sess137_papers_411@linklings.com SUMMARY:Portrait Video Editing Empowered by Multimodal Generative Priors DESCRIPTION:Technical Papers\n\nXuan Gao, Haiyao Xiao, Chenglai Zhong, Shi min Hu, Yudong Guo, and Juyong Zhang (University of Science and Technology of China)\n\nWe introduce PortraitGen, a powerful portrait video editing method that achieves consistent and expressive stylization with multimodal prompts. Traditional portrait video editing methods often struggle with 3 D and temporal consistency, and typically lack in rendering quality and ef ficiency. To address these issues, we lift the portrait video frames to a unified dynamic 3D Gaussian field, which ensures structural and temporal c oherence across frames. Furthermore, we design a novel Neural Gaussian tex ture mechanism that not only enables sophisticated style editing but also achieves rendering speed over 100FPS. Our approach incorporates multimodal inputs through knowledge distilled from large-scale 2D generative models. Our system also incorporates expression similarity guidance and a face-aw are portrait editing module, effectively mitigating degradation issues ass ociated with iterative dataset updates. Extensive experiments demonstrate the temporal consistency, editing efficiency, and superior rendering quali ty of our method. The broad applicability of the proposed approach is demo nstrated through various applications, including text-driven editing, imag e-driven editing, and relighting, highlighting its great potential to adva nce the field of video editing.\n\nRegistration Category: Full Access, Ful l Access Supporter\n\nLanguage Format: English Language\n\nSession Chair: Michael Rubinstein (Google) URL:https://asia.siggraph.org/2024/program/?id=papers_411&sess=sess137 END:VEVENT END:VCALENDAR