Past events

Paritosh Desai (Google Inc.)

Registration is required to access the Zoom webinar.

While there are many commonalities between academic research and roles in the industry for applied math professionals, there are also important differences. These differences are material in shaping career outcomes in the industry and we try to elaborate on them by focusing on two broad themes for people with academic research backgrounds. First, we will look at the common patterns related to applied AI/ML problems across multiple industries and specific challenges around them. Second, we will discuss emergent requirements for success in the industry setting. We will share principles and anecdotes related to data, software engineering practices, and empirical research based upon industry experiences.

Alex Gittens (Rensselaer Polytechnic Institute)

Registration is required to access the Zoom webinar. Alex will also be presenting in person in Walter 402.

In the context of numerical linear algebra algorithms, where it is natural to sacrifice accuracy in return for quicker computation of solutions whose errors are only slightly larger than optimal, the time-accuracy tradeoff of randomized sketching has been well-characterized. Algorithms such as Blendenpik and LSRN have shown that carefully designed randomized algorithms can outperform industry standard linear algebra codes such as those provided in LAPACK.
For numerical tensor algorithms, where the size of problems grow exponentially with the order of the tensor, it is even more desirable to use randomization. However, in this setting, the time-accuracy tradeoff of randomized sketching is more difficult to understand and exploit, as:

(1) in the first place, tensor problems are non-convex, 
(2) the properties of the data change from iteration to iteration, and
(3) straightforward applications of standard results on randomized sketching allow for the error to increase from iteration to iteration.

On the other hand, the iterative nature of such algorithms opens up the opportunity to learn how to sketch more accurately in an online manner.

In this talk we consider the problem of speeding up the computation of low CP-rank (canonical polyadic) approximations of tensors through regularized sketching. We establish for the first time a sublinear convergence rate to approximate critical points of the objective under standard conditions, and further provide algorithms that adaptively select the sketching and regularization rates.

Alex Gittens is an assistant professor of computer science at Rensselaer Polytechnic Institute. He obtained his PhD in applied mathematics from CalTech in 2013, and BSes in mathematics and electrical engineering from the University of Houston. After his PhD, he joined the eBay machine learning research group, then the AMPLab (now the RISELab) at UC Berkeley, before joining RPI. His research interests lie at the intersection of randomized linear algebra and large-scale machine learning, in particular encompassing nonlinear and multilinear low-rank approximations; sketching for nonlinear and multilinear problems; and scalable and data-dependent kernel learning.

The last day to receive a 100% tuition refund for canceling full semester classes is Monday, January 24.

View the full academic schedule on One Stop.
 

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Thursday from 12 p.m. - 1 p.m. during the Fall 2021 semester.

This week's speaker, Yihe Dong (Google), will be giving a talk titled "Attention is not all you need."

Abstract

I will be talking about our recent work on better understanding attention. Attention-based architectures have become ubiquitous in machine learning, yet our understanding of the reasons for their effectiveness remains limited. We show that self-attention possesses a strong inductive bias towards "token uniformity". Specifically, without skip connections or multi-layer perceptrons (MLPs), the output converges doubly exponentially to a rank-1 matrix. On the other hand, skip connections and MLPs stop the output from degeneration. Along the way, we develop a useful decomposition of attention architectures. This is joint work with Jean-Baptiste Cordonnier and Andreas Loukas.

Biography

Yihe Dong is a machine learning researcher and engineer at Google, with interests in geometric deep learning and natural language processing.

Welcome back! The spring 2022 semester begins on Tuesday, January 18.

View the full academic schedule on One Stop.
 

The University of Minnesota will be closed in observance of Martin Luther King, Jr. Day.

View the full schedule of University holidays.
 

The University of Minnesota will be closed in observance of New Year's Day.

View the full schedule of University holidays.
 

The application deadline for the data science majors is December 30.

Undergraduate students typically apply to a major while enrolled in fall semester courses during their sophomore year (third semester).

Submit your application here.

All applicants will be notified of their admission decision via email within three weeks of the application deadline.

The University of Minnesota will be closed (floating holiday).

View the full schedule of University holidays.
 

The University of Minnesota will be closed in observance of Christmas Day.

View the full schedule of University holidays.