BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:18871231T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20250110T023312Z
LOCATION:Hall B7 (1)\, B Block\, Level 7
DTSTART;TZID=Asia/Tokyo:20241205T132800
DTEND;TZID=Asia/Tokyo:20241205T134200
UID:siggraphasia_SIGGRAPH Asia 2024_sess132_papers_342@linklings.com
SUMMARY:CBIL: Collective Behavior Imitation Learning for Fish from Real Vi
 deos
DESCRIPTION:Technical Papers\n\nYifan Wu (University of Hong Kong); Zhiyan
 g Dou (University of Hong Kong, University of Pennsylvania); Yuko Ishiwaka
  and Shun Ogawa (SoftBank); Yuke Lou (University of Hong Kong); Wenping Wa
 ng (Texas A&M University); Lingjie Liu (University of Pennsylvania); and T
 aku Komura (University of Hong Kong)\n\nReproducing realistic collective b
 ehaviors presents a captivating yet formidable challenge. Traditional rule
 -based methods rely on hand-crafted principles, limiting motion diversity 
 and realism in generated collective behaviors. Recent imitation learning m
 ethods learn from data but often require ground truth motion trajectories 
 and struggle with authenticity, especially in high-density groups with err
 atic movements. In this paper, we present a scalable approach, Collective 
 Behavior Imitation Learning (CBIL), for learning fish schooling behavior d
 irectly from videos, without relying on captured motion trajectories. Our 
 method first leverages Video Representation Learning, where a Masked Video
  AutoEncoder (MVAE) extracts implicit states from video inputs in a self-s
 upervised manner. The MVAE effectively maps 2D observations to implicit st
 ates that are compact and expressive for following the imitation learning 
 stage. Then, we propose a novel adversarial imitation learning method to e
 ffectively capture complex movements of the schools of fish, allowing for 
 efficient imitation of the distribution for motion patterns measured in th
 e latent space. It also incorporates bio-inspired rewards alongside priors
  to regularize and stabilize training. Once trained, CBIL can be used for 
 various animation tasks with the learned collective motion priors. We furt
 her show its effectiveness across different species. Finally, we demonstra
 te the application of our system in detecting abnormal fish behavior from 
 in-the-wild videos.\n\nRegistration Category: Full Access, Full Access Sup
 porter\n\nLanguage Format: English Language\n\nSession Chair: Yi Zhou (Ado
 be)
URL:https://asia.siggraph.org/2024/program/?id=papers_342&sess=sess132
END:VEVENT
END:VCALENDAR