BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260817T171534Z
LOCATION:Hall B7 (1)\, B Block\, Level 7
DTSTART;TZID=Asia/Tokyo:20241205T132800
DTEND;TZID=Asia/Tokyo:20241205T134200
UID:siggraphasia_SIGGRAPH Asia 2024_sess132_papers_342@linklings.com
SUMMARY:CBIL: Collective Behavior Imitation Learning for Fish from Real Vi
 deos
DESCRIPTION:Yifan Wu (University of Hong Kong); Zhiyang Dou (University of
  Hong Kong, University of Pennsylvania); Yuko Ishiwaka and Shun Ogawa (Sof
 tBank); Yuke Lou (University of Hong Kong); Wenping Wang (Texas A&M Univer
 sity); Lingjie Liu (University of Pennsylvania); and Taku Komura (Universi
 ty of Hong Kong)\n\nReproducing realistic collective behaviors presents a 
 captivating yet formidable challenge. Traditional rule-based methods rely 
 on hand-crafted principles, limiting motion diversity and realism in gener
 ated collective behaviors. Recent imitation learning methods learn from da
 ta but often require ground truth motion trajectories and struggle with au
 thenticity, especially in high-density groups with erratic movements. In t
 his paper, we present a scalable approach, Collective Behavior Imitation L
 earning (CBIL), for learning fish schooling behavior directly from videos,
  without relying on captured motion trajectories. Our method first leverag
 es Video Representation Learning, where a Masked Video AutoEncoder (MVAE) 
 extracts implicit states from video inputs in a self-supervised manner. Th
 e MVAE effectively maps 2D observations to implicit states that are compac
 t and expressive for following the imitation learning stage. Then, we prop
 ose a novel adversarial imitation learning method to effectively capture c
 omplex movements of the schools of fish, allowing for efficient imitation 
 of the distribution for motion patterns measured in the latent space. It a
 lso incorporates bio-inspired rewards alongside priors to regularize and s
 tabilize training. Once trained, CBIL can be used for various animation ta
 sks with the learned collective motion priors. We further show its effecti
 veness across different species. Finally, we demonstrate the application o
 f our system in detecting abnormal fish behavior from in-the-wild videos.\
 n\nRegistration Category: Full Access, Full Access Supporter\n\nLanguage F
 ormat: English Language\n\nSession Chair: Yi Zhou (Roblox)\n\n
URL:https://asia.siggraph.org/2024/program/?id=papers_342&sess=sess132
END:VEVENT
END:VCALENDAR