CS&E Colloquium: No such thing as a model-free lunch? Model-free search and reliable decision making

The computer science colloquium takes place on Mondays and Fridays from 11:15 a.m. - 12:15 p.m.

This week's speaker, Jack Umenberger (MIT), will be giving a talk titled "No such thing as a model-free lunch? Model-free search and reliable decision making".

Abstract

Inspired by breakthrough results in the processing of complex data, machine learning is being increasingly applied to problems in decision-making and control. However, to be suitable for deployment in applications, we require learning-based algorithms that come with guarantees of reliability, robustness, and safety. 

This talk will focus on the reliability of model-free policy search, addressing the question: when do such methods find optimal solutions, and when do they get trapped in poor local minima? Existing work has considered static policies; however, for dynamic policies that remember past observations - necessary for optimal decision making in many applications - these questions have hitherto remained unanswered. Focusing on the classic control-theoretic problem of output estimation, I will present the first model-free policy search algorithm for dynamic policies guaranteed to converge to the optimal solution. 

Along the way, I’ll also describe my path toward working on this problem, highlighting some of my contributions to model-based approaches for safe and reliable control, including data-driven robust control, system identification, and trajectory optimization. I will offer my perspective on the strengths and weaknesses of model-free and model-based methods, as well as the ways in which they complement each other. 

The talk will conclude by discussing the potential of harnessing the best of model-free and model-based approaches for tackling challenging optimization problems more broadly, including those involving a mixture of continuous and discrete decisions. 

Biography

Jack Umbenberger is a postdoctoral associate in Russ Tedrake's Robot Locomotion at the Massachusetts Institute of Technology. He received his PhD in Engineering and B.E. in Mechatronics from The University of Sydney, Australia, in 2018 and 2013, respectively, and was a postdoctoral fellow in the Division of Systems and Control at Uppsala University, Sweden, from 2017-2019. He is interested in understanding how learning can improve decision making in uncertain and complex environments, with a focus on modeling and control of dynamical systems from data.