BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171531Z
LOCATION:Hall B5 (2)\, B Block\, Level 5
DTSTART;TZID=Asia/Tokyo:20241205T154300
DTEND;TZID=Asia/Tokyo:20241205T155400
UID:siggraphasia_SIGGRAPH Asia 2024_sess134_papers_785@linklings.com
SUMMARY:TrailBlazer: Trajectory Control for Diffusion-Based Video Generati
 on
DESCRIPTION:Wan-Duo Kurt Ma (Victoria University of Wellington), J. P. Lew
 is (NVIDIA Research), and W. Bastiaan Kleijn (Victoria University of Welli
 ngton)\n\nLarge text-to-video (T2V) models such as Sora have the potential
  to revolutionize visual effects and the creation of some types of movies.
  Current T2V models require tedious trial-and-error experimentation to ach
 ieve desired results, however. This motivates the search for methods to di
 rectly control desired attributes. In this work, we take a step toward thi
 s goal, introducing a method for high-level, temporally-coherent control o
 ver the basic trajectories and appearance of objects. Our algorithm, Trail
 Blazer, allows the general positions and (optionally) appearance of object
 s to be controlled simply by keyframing approximate bounding boxes and (op
 tionally) their corresponding prompts. Importantly, our method does not re
 quire a pre-existing control video signal that already contains an accurat
 e outline of the desired motion, yet the synthesized motion is surprisingl
 y natural with emergent effects including perspective and movement toward 
 the virtual camera as the box size increases. The method is efficient, mak
 ing use of a pre-trained T2V model and requiring no training or fine-tunin
 g, with negligible additional computation. Specifically, the bounding box 
 controls are used as soft masks to guide manipulation of the self-attentio
 n and cross-attention modules in the video diffusion model. While our visu
 al results are limited by those of the underlying model, the algorithm may
  generalize to future models that use standard self- and cross-attention c
 omponents.\n\nRegistration Category: Full Access, Full Access Supporter\n\
 nLanguage Format: English Language\n\nSession Chair: Nanxuan Zhao (Adobe R
 esearch)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_785&sess=sess134
END:VEVENT
END:VCALENDAR