Past events

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Wednesday from 11 a.m. - 12 p.m. during the Fall 2022 semester.

This week's speaker, Sijia Liu (Michigan State University), will be giving a talk titled "Robust and Efficient Neural Network Training: A New Bi-level Learning Paradigm".

Abstract

This talk will introduce Bi-level Machine Learning (BML), an emerging but overlooked topic rooted in bi-level optimization (BLO), to tackle the recent neural network training challenges in robust and efficient AI. In the first part, I will revisit adversarial training (AT)–a widely recognized training mechanism to gain adversarial robustness of deep neural networks–from a fresh BML viewpoint. Built upon that, I will introduce a new theoretically-grounded and computationally-efficient robust training algorithm termed Fast Bi-level AT (Fast-BAT), which can defend sign-based projected gradient descent (PGD) attacks without using any gradient sign method or explicit robust regularization.

In the second part, I will move to a sparse learning paradigm that aims at pruning large-scale neural networks for improved generalization and efficiency. As demonstrated by the Lottery Ticket Hypothesis (LTH), iterative magnitude pruning (IMP) is the predominant sparse learning method to successfully find ‘winning’ sparse sub-networks. Yet, the computation cost of IMP grows prohibitively as the sparsity ratio increases. I will show that BML provides a graceful algorithmic foundation for model pruning and helps us close the gap between pruning accuracy and efficiency. Please see the references and codes at Sijia Liu's GitHub repository.

Biography

Dr. Sijia Liu is currently an Assistant Professor at the Department of Computer Science and Engineering, Michigan State University, and an Affiliated Professor at the MIT-IBM Watson AI Lab, IBM Research. His research spans the areas of machine learning, optimization, computer vision, signal processing, and computational biology, with a recent focus on Trustworthy and Scalable ML. He received the Best Paper Runner-Up Award at the Conference on Uncertainty in Artificial Intelligence (UAI) in 2022 and the Best Student Paper Award at the 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) in  2017. He has published over 60 papers at top-tier ML/AI conferences and (co-)organized several tutorials and workshops on Trustworthy AI and Optimization for Deep Learning at KDD, AAAI, CVPR, and ICML, to name a few.

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Wednesday from 11 a.m. - 12 p.m. during the Fall 2022 semester.

This week's speaker, Yifan Peng (PHS, Cornell U.), will be giving a talk.

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Wednesday from 11 a.m. - 12 p.m. during the Fall 2022 semester.

This week's speaker, Juggy Jagannathan (3M health), will be giving a talk titled "AI and Future of Healthcare."

Abstract

AI technology is permeating every aspect of healthcare. It is transforming how care is provided in acute, ambulatory and home settings. It is transforming how population health is being managed. AI in combination with other disciplines, such as genetics and virtual reality, is accelerating innovative treatments. Drug discovery and clinical trials are being transformed. This presentation will provide an overview of where AI is already being used and prognosis for the immediate future.

Biography

V. “Juggy” Jagannathan, PhD, researches artificial intelligence and computer science for 3M Health Information Systems, developing new natural language and deep learning technologies that automatically structure patient-physician conversations into clinical documents. As an AI researcher with over four decades in the field, Juggy’s background includes both industry technology development and academia. The company he founded in 1996 with technology from West Virginia University (WVU) was acquired by M*Modal in 2003 and he has been with the company ever since. He is also an adjunct professor teaching Computer Science and Natural Language Processing (NLP) at WVU. Juggy is a tennis nut, golf nut and a yoga nut. In short, a nut.

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Thursday from 11 a.m. - 12 p.m. during the Fall 2022 semester.

This week's speaker, Aryan Mokhtari (UT Austin), will be giving a talk titled "The power of adaptivity in representation learning".

Abstract

From meta-learning to federated learning

A central problem in machine learning is as follows: How should we train models using data generated from a collection of clients/environments, if we know that these models will be deployed in a new and unseen environment?

In the setting of few-shot learning, two prominent approaches are: (a) develop a modeling framework that is “primed” to adapt, such as Model Adaptive Meta Learning (MAML), or (b) develop a common model using federated learning (such as FedAvg), and then fine tune the model for the deployment environment. We study both these approaches in the multi-task linear representation setting. We show that the reason behind generalizability of the models in new environments trained through either of these approaches is that the dynamics of training induces the models to evolve toward the common data representation among the clients’ tasks.

In both cases, the structure of the bi-level update at each iteration (an inner and outer update with MAML, and a local and global update with FedAvg) holds the key — the diversity among client data distributions are exploited via inner/local updates, and induces the outer/global updates to bring the representation closer to the ground-truth. In both these settings, these are the first results that formally show representation learning, and derive exponentially fast convergence to the ground-truth representation. Based on joint work with Liam Collins, Hamed Hassani, Sewoong Oh, and Sanjay Shakkottai.

Biography

Aryan Mokhtari is an Assistant Professor in the Electrical and Computer Engineering Department of the University of Texas at Austin (UT Austin) where he holds the Fellow of Texas Instruments/Kilby. Before joining UT Austin, he was a Postdoctoral Associate in the Laboratory for Information and Decision Systems (LIDS) at MIT.  Prior to that, he was a Research Fellow at the Simons Institute for the program on “Bridging Continuous and Discrete Optimization”. He received his Ph.D. in electrical and systems engineering from the University of Pennsylvania (Penn). He is the recipient of the Army Research Office (ARO) Early Career Program Award, the Simons-Berkeley Research Fellowship, and Penn’s Joseph and Rosaline Wolf Award for Best Doctoral Dissertation.

RSVP today!.

During each session, the graduate staff will review:

• Requirements (general)
• Applying
• Prerequisite requirements
• What makes a strong applicant
• Funding
• Resources
• Common questions
• Questions from attendees

Students considering the following programs should attend:

• Computer Science M.S.
• Computer Science M.C.S.
• Computer Science Ph.D.
• Data Science M.S.
• Data Science Post-Baccalaureate Certificate
  
   

Abstract

We develop an assisted learning framework for assisting organization-level learners in improving their learning performance with limited and imbalanced data. In particular, learners at the organizational level usually have sufficient computation resource, but are subject to stringent collaboration policy and information privacy. Their limited imbalanced data often cause biased inference and sub-optimal decision-making. In our assisted learning framework, an organizational learner purchases assistance service from a service provider and aims to enhance its model performance within a few assistance rounds. We develop effective stochastic training algorithms for assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to frequently transmit gradients or models, our framework allows the learner to only occasionally share information with the service provider, and still achieve a near-oracle model as if all the data were centralized.
 

Biography

Yi Zhou is an assistant professor affiliated with the Department of ECE at the University of Utah. Before, he worked as a post-doctorate research associate in the Department of ECE at Duke University. He obtained a Ph.D. degree in ECE from The Ohio State University in 2018. His research interests include deep learning, reinforcement learning, statistical machine learning, nonconvex and distributed optimization, and statistical signal processing.

You may attend the talk either in person in Walter 402 or register via Zoom. Registration is required to access the Zoom webinar.

Title: Does the Data Induce Capacity Control in Deep Learning?

Abstract: Accepted statistical wisdom suggests that larger the model class, the more likely it is to overfit the training data. And yet, deep networks generalize extremely well. The larger the deep network, the better its accuracy on new data. This talk seeks to shed light upon this apparent paradox.

We will argue that deep networks are successful because of a characteristic structure in the space of learning tasks. The input correlation matrix for typical tasks has a peculiar (“sloppy”) eigenspectrum where, in addition to a few large eigenvalues (salient features), there are a large number of small eigenvalues that are distributed uniformly over exponentially large ranges. This structure in the input data is strongly mirrored in the representation learned by the network. A number of quantities such as the Hessian, the Fisher Information Matrix, as well as others activation correlations and Jacobians, are also sloppy. Even if the model class for deep networks is very large, there is an exponentially small subset of models (in the number of data) that fit such sloppy tasks. This talk will demonstrate the first analytical non-vacuous generalization bound for deep networks that does not use compression. We will also discuss an application of these concepts that develops new algorithms for semi-supervised learning.

Bio:
Pratik Chaudhari is an Assistant Professor in Electrical and Systems Engineering and Computer and Information Science at the University of Pennsylvania. He is a member of the GRASP Laboratory. From 2018-19, he was a Senior Applied Scientist at Amazon Web Services and a Postdoctoral Scholar in Computing and Mathematical Sciences at Caltech. Pratik received his PhD (2018) in Computer Science from UCLA, his Master's (2012) and Engineer's (2014) degrees in Aeronautics and Astronautics from MIT. He was a part of NuTonomy Inc. (now Hyundai- Aptiv Motional) from 2014—16. He received the NSF CAREER award and the Intel Rising Star Faculty Award in 2022.

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Wednesday from 11 a.m. - 12 p.m. during the Fall 2022 semester.

This week's speaker, Hong Pang (UMN Department of Pharmaceutics), will be giving a talk titled "Peptide Targeting for Novel Therapies and Cell Biology."

Biography

Dr. Pang is an assistant professor at the Department of Pharmaceutics, University of Minnesota. He graduated from University of Utah as a PhD of biochemistry, and did postdoc training at Sanford-Burnham-Prebys institute at La Jolla, CA.

His lab is specialized in developing peptide-based targeted therapeutics, and eliciting the cellular transport processes. Using phage display, his lab is screening for peptides targeting any given disease or cell type, and deciphering the underlying biomarkers. Through chemical modification and nanotechnology, his lab applies targeting peptides to improve the delivery efficiency and specificity of therapeutic and diagnostic payloads. Moreover, he is interested in the endocytosis, exocytosis and intercellular communication triggered by peptides and their payloads. Using simulation and experimental approaches, his group is investigating the biophysical and molecular basis of these transport processes.

RSVP today!.

During each session, the graduate staff will review:

• Requirements (general)
• Applying
• Prerequisite requirements
• What makes a strong applicant
• Funding
• Resources
• Common questions
• Questions from attendees

Students considering the following programs should attend:

• Computer Science M.S.
• Computer Science M.C.S.
• Computer Science Ph.D.
• Data Science M.S.
• Data Science Post-Baccalaureate Certificate