UMN Machine Learning Seminar: Diametrical Risk Minimization - Theory and Computations

The UMN Machine Learning Seminar Series brings together faculty, students, and local industrial partners who are interested in the theoretical, computational, and applied aspects of machine learning, to pose problems, exchange ideas, and foster collaborations. The talks are every Thursday from 12 p.m. - 1 p.m. during the Fall 2021 semester.

This week's speaker, Johannes Royset (Naval Postgraduate School), will be giving a talk titled "Diametrical Risk Minimization: Theory and Computations."

Abstract

The theoretical and empirical performance of Empirical Risk Minimization (ERM) often suffers when loss functions are poorly behaved with large Lipschitz moduli and spurious sharp minimizers. We propose and analyze a counterpart to ERM called Diametrical Risk Minimization (DRM), which accounts for worst-case empirical risks within neighborhoods in parameter space. DRM has generalization bounds that are independent of Lipschitz moduli for convex as well as nonconvex problems and it can be implemented using a practical algorithm based on stochastic gradient descent. Numerical results illustrate the ability of DRM to find quality solutions with low generalization error in sharp empirical risk landscapes from benchmark neural network classification problems with corrupted labels.

Biography

Dr. Johannes O. Royset is Professor of Operations Research at the Naval Postgraduate School. Dr. Royset's research focuses on formulating and solving stochastic and deterministic optimization problems arising in data analytics, sensor management, and reliability engineering. He was awarded a National Research Council postdoctoral fellowship in 2003, a Young Investigator Award from the Air Force Office of Scientific Research in 2007, and the Barchi Prize as well as the MOR Journal Award from the Military Operations Research Society in 2009. He received the Carl E. and Jessie W. Menneken Faculty Award for Excellence in Scientific Research in 2010 and the Goodeve Medal from the Operational Research Society in 2019. Dr. Royset was a plenary speaker at the International Conference on Stochastic Programming in 2016 and at the SIAM Conference on Uncertainty Quantification in 2018. He has a Doctor of Philosophy degree from the University of California at Berkeley (2002). Dr. Royset has been an associate or guest editor of SIAM Journal on Optimization, Operations Research, Mathematical Programming, Journal of Optimization Theory and Applications, Naval Research Logistics, Journal of Convex Analysis, Set-Valued and Variational Analysis, and Computational Optimization and Applications. He is the author of about 100 papers and two books.

Start date
Thursday, Nov. 11, 2021, Noon
End date
Thursday, Nov. 11, 2021, 1 p.m.
Location

Share