ML Seminar: Sijia Liu
The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Wednesday from 11 a.m. - 12 p.m. during the Fall 2022 semester.
This week's speaker, Sijia Liu (Michigan State University), will be giving a talk titled "Robust and Efficient Neural Network Training: A New Bi-level Learning Paradigm".
This talk will introduce Bi-level Machine Learning (BML), an emerging but overlooked topic rooted in bi-level optimization (BLO), to tackle the recent neural network training challenges in robust and efficient AI. In the first part, I will revisit adversarial training (AT)–a widely recognized training mechanism to gain adversarial robustness of deep neural networks–from a fresh BML viewpoint. Built upon that, I will introduce a new theoretically-grounded and computationally-efficient robust training algorithm termed Fast Bi-level AT (Fast-BAT), which can defend sign-based projected gradient descent (PGD) attacks without using any gradient sign method or explicit robust regularization.
In the second part, I will move to a sparse learning paradigm that aims at pruning large-scale neural networks for improved generalization and efficiency. As demonstrated by the Lottery Ticket Hypothesis (LTH), iterative magnitude pruning (IMP) is the predominant sparse learning method to successfully find ‘winning’ sparse sub-networks. Yet, the computation cost of IMP grows prohibitively as the sparsity ratio increases. I will show that BML provides a graceful algorithmic foundation for model pruning and helps us close the gap between pruning accuracy and efficiency. Please see the references and codes at Sijia Liu's GitHub repository.
Dr. Sijia Liu is currently an Assistant Professor at the Department of Computer Science and Engineering, Michigan State University, and an Affiliated Professor at the MIT-IBM Watson AI Lab, IBM Research. His research spans the areas of machine learning, optimization, computer vision, signal processing, and computational biology, with a recent focus on Trustworthy and Scalable ML. He received the Best Paper Runner-Up Award at the Conference on Uncertainty in Artificial Intelligence (UAI) in 2022 and the Best Student Paper Award at the 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) in 2017. He has published over 60 papers at top-tier ML/AI conferences and (co-)organized several tutorials and workshops on Trustworthy AI and Optimization for Deep Learning at KDD, AAAI, CVPR, and ICML, to name a few.