CS&E Colloquium: Self-supervised Behavioral Imaging
The computer science colloquium takes place on Mondays from 11:15 a.m. - 12:15 p.m.
This week's speaker, Hyun Soo Park (University of Minnesota), will be giving a talk titled "Self-supervised Behavioral Imaging."
Abstract
Nonverbal behavioral signals such as gaze direction, facial expression, and body gesture have been ingrained into our interpersonal communications, which often appear at microscopic scale. Despite their omnipresence in all aspects of social interactions, existing AI systems are nearly blinded to them. In this talk, I will walk through our effort towards enabling 3D behavior imaging---a computational model that allows precise measurements of microscopic social signals from numerous multiview cameras. A key challenge is that social interactions inherently induce self-occlusion, which fundamentally limits accurate 3D reconstruction from the image streams. I will argue that associating semantic meaning with geometry, e.g., holistic finger pose, provides a strong cue to predict the missing data. To learn such visual semantics, I will introduce a new large-scale human behavior dataset called HUMBI scanned by 107 HD cameras at Minnesota State Fair. In the second part of the talk, I will discuss a computational approach to measure free-ranging behaviors of monkeys for neuroscience study. Unlike humans, these monkeys are challenging due to lack of annotation data. To address this, I will introduce a semi-supervised learning framework that leverages multiview geometry and tracking to reconstruct their motion in 3D.
Biography
Hyun Soo Park is an Assistant Professor at the Department of Computer Science & Engineering, the University of Minnesota (UMN). He is interested in computer vision approaches for behavioral imaging. He has received NSF's CRII, NSF's CAREER Awards, and CVPR 2021 Best Paper Honorable Mention Award. Prior to UMN, he was a Postdoctoral Fellow in GRASP Lab at University of Pennsylvania. He earned his Ph.D. from Carnegie Mellon University.
