Past Events

Industrial Problems Seminar

Katy Micek (Paramount)

Abstract

Over the past decade, there has been an enormous amount of buzz and enthusiasm about data science. Consider, for example, the alluring title of a Harvard Business Review’s article published in October 2012: “Data Scientist: The Sexiest Job of the 21st Century.” This article was published shortly after I started working in the data science space, and I’ve had a front-row seat to the field’s evolution since that time. The goal of my talk is to provide a perspective on what a career in data science is like. I’ll start by sharing my experience working in the field across various industries and how technical roles have changed during that time. I will then explain how organizations seek to utilize data science and provide examples of the challenges (technical and organizational) that arise while implementing solutions. Finally, I will offer strategies for finding a job that is a good fit in the broadly defined, rapidly changing field of data science.

Data Science Seminar

Konstantinos Spiliopoulos (Boston University)

Abstract

We study the effect of normalization on the layers of deep neural networks. A given layer $i$ with $N_{i}$ hidden units is allowed to be normalized by $1/N_{i}^{\gamma_{i}}$ with $\gamma_{i}\in[1/2,1]$ and we study the effect of the choice of the $\gamma_{i}$ on the statistical behavior of the neural network’s output (such as variance) as well as on the test accuracy on the MNIST and CIFAR10 data sets. We find that in terms of variance of the neural network’s output and test accuracy the best choice is to choose the $\gamma_{i}$’s to be equal to one, which is the mean-field scaling. We also find that this is particularly true for the outer layer, in that the neural network’s behavior is more sensitive in the scaling of the outer layer as opposed to the scaling of the inner layers. The mechanism for the mathematical analysis is an asymptotic expansion for the neural network’s output and corresponding mean field analysis. An important practical consequence of the analysis is that it provides a systematic and mathematically informed way to choose the learning rate hyperparameters. Such a choice guarantees that the neural network behaves in a statistically robust way as the $N_i$'s grow to infinity.

Industrial Problems Seminar

Vasileios Ioannidis (Amazon Search AI)

Please note the 10:10am start time.

Abstract

Graph neural networks (GNNs) learn from complex graph data and have been remarkably successful in various applications and across industries. This presentation first introduces GNNs via the message passing framework and dives into popular GNN variants. Next, it explores the fusion of textual data with heterogeneous graph structures to improve semantic and behavioral representations. It introduces the Language Model GNN (LM-GNN), a framework that efficiently combines large language models and Graph Neural Networks (GNNs) through fine-tuning. LM-GNN supports various tasks like node classification and link prediction and demonstrates its effectiveness. Another aspect addressed is the challenge of effective node representation learning in textual graphs. The Graph-Aware Distillation (Grad) framework is proposed, which encodes graph structures into a Language Model (LM) to enable fast and scalable inference. Grad optimizes GNN and a graphless student model, resulting in superior performance in node classification tasks. Finally, the presentation discusses pre-training text and graph models on large, heterogeneous graphs with textual data using the Graph-Aware Language Model Pre-Training (GALM) framework. It highlights the framework's effectiveness through experiments on real datasets.

Industrial Problems Seminar 

Parker Williams (Rivian Automotive)

Abstract

Autoencoders are a type of network designed to learn efficient encodings of data, typically for purposes of unsupervised data compression. I will outline a process to leverage autoencoders for unsupervised anomaly detection, which has become an essential tool in edge based system health monitoring. I will begin with a naive implementation and motivate an autoencoder variation from an anomaly detection perspective. We will then go through a few examples and implementation challenges encountered in the wild. We will end with broader observations on when this methodology can be effective and lessons learned from an organizational and software engineering perspective.

Data Science Seminar

Dongbin Xiu (The Ohio State University)

Abstract

We present a framework of predictive modeling of unknown systems from measurement data. The method is designed to discover/approximate the unknown evolution operator, i.e., flow map, behind the data. Deep neural network (DNN) is employed to construct such an approximation. Once an accurate DNN model for the evolution operator is constructed, it serves as a predictive model for the unknown system and enables us to conduct system analysis. We demonstrate that flow map learning (FML) approach is applicable for modeling a wide class of problems, including dynamical systems, systems with missing variables and hidden parameters, as well as partial differential equations (PDEs).

Industrial Problems Seminar 

Christopher Bemis (X Cubed Capital Management)

Abstract

We study the effect linear constraints have on risk in the context of mean variance optimization (MVO). Jagannathan and Ma (2003) establish an equivalence between certain constrained and unconstrained MVO problems via a modification of the covariance matrix. We extend their results to arbitrary linear constraints and provide alternative interpretations for the effect of constraints on both the input parameters to the problems at hand and why ex-post performance is improved in the constrained setting.  In addition, we present a signal modification strategy similar in approach to that of Black-Litterman.

Data Science Seminar

Alex Gittens (Rensselaer Polytechnic Institute)

Abstract

Classical linear algebraic algorithms guarantee high accuracy in exchange for high computational cost. These costs can be infeasible in modern applications, so over the last two decades, randomized algorithms have been developed that allow a user-specified trade-off between accuracy and computational efficiency when dealing with massive data sets. The intuition is that when dealing with an excess of structured data (e.g., a large matrix which has low numerical rank), one can toss away a large portion of this data, thereby reducing the computational load, without introducing much additional error into the computation. In this talk we look at the design and performance analysis of several numerical linear algebra and machine learning algorithms--- including linear solvers, approximate kernel machines, and tensor low-rank decomposition--- based upon this principle.

Data Science Seminar

Jiajia Yu (Duke University)

Abstract

Mean-field games study the behavior of a large number of rational agents in a non-cooperative game. It has wide applications in various fields. But it is not easy to solve the mean-field game numerically because of its complicated structure.

In the first part of my talk, I will present an efficient and flexible algorithm for dynamic mean-field games. The algorithm is based on an accelerated proximal gradient method. It consists of an easy-to-implement gradient descent step and a projection step equivalent to solving an elliptic equation. We also extend the setting of mean-field games and the algorithm to manifolds. In the second part of my talk, I will bridge mean-field games with a deep generative model which is called normalizing flows. The connection gives a computational approach for high-dimensional mean-field games and improves the training of the generative model.

The first part is based on joint works with Rongjie Lai (Purdue), Wuchen Li (UofSC) and Stanley Osher (UCLA). The second part is based on a joint work with Han Huang (RPI), Rongjie Lai (Purdue) and Jie Chen (IBM).