CSE DSI Machine Learning Seminar with Albert Berahas (University of Michigan)

Next Generation Algorithms for Stochastic Optimization with Constraints

Stochastic gradient and related methods for solving stochastic optimization problems have been studied extensively in recent years. It has been shown that such algorithms and much of their convergence and complexity guarantees extend in straightforward ways when one considers problems involving simple constraints, such as when one can perform projections onto the feasible region of the problem. However, settings with general nonlinear constraints have received less attention, and many of the approaches that have been proposed for solving such problems resort to using penalty or (augmented) Lagrangian methods, which are often not the most effective strategies. In this work, we propose and analyze stochastic optimization algorithms for deterministically constrained problems based on the sequential quadratic optimization (commonly known as SQP) methodology. We discuss the rationale behind our proposed techniques, convergence in expectation and complexity guarantees for our algorithms, and present numerical experiments that we have performed. This is joint work with Raghu Bollapragada, Frank E. Curtis, Michael O'Neill, Daniel P. Robinson, Jiahao Shi and Baoyu Zhou.

Albert S. Berahas is an Assistant Professor in the Industrial and Operations Engineering department at the University of Michigan. Before joining the University of Michigan, he was a Postdoctoral Research Fellow in the Industrial and Systems Engineering department at Lehigh University working with Professors Katya Scheinberg, Frank Curtis and Martin Takáč. Prior to that appointment, he was a Postdoctoral Research Fellow in the Industrial Engineering and Management Sciences department at Northwestern University working with Professor Jorge Nocedal. Berahas completed his PhD studies in the Engineering Sciences and Applied Mathematics (ESAM) department at Northwestern University in 2018, advised by Professor Jorge Nocedal. He received his undergraduate degree in Operations Research and Industrial Engineering (ORIE) from Cornell University in 2009, and in 2012 obtained an MS degree in Applied Mathematics from Northwestern University. Berahas’ research broadly focuses on designing, developing and analyzing algorithms for solving large scale nonlinear optimization problems. Specifically, he is interested in and has explored several sub-fields of nonlinear optimization such as: (i) general nonlinear optimization algorithms, (ii) optimization algorithms for machine learning, (iii) constrained optimization, (iv) stochastic optimization, (v) derivative-free optimization, and (vi) distributed optimization. Berahas is served as the vice-chair of the Nonlinear Optimization cluster for the INFORMS Optimization Society (2020-2022), the chair of the Nonlinear Optimization cluster for the INFORMS Optimization Society Conference (2021-2022), and the co-chair of the Nonlinear Optimization cluster for the ICCOPT 2022 conference (2021-2022). Berahas is the president of the INFORMS Junior Faculty Interest Group (JFIG).