Combining Learning and Control in Cyber-Physical Systems
A Warren Distinguished Lecture with Andreas Malikopoulos, Civil & Environmental Engineering, Cornell University
Abstract
Cyber-physical systems (CPS), in most instances, represent systems of subsystems with an informationally decentralized structure. To derive optimal control strategies for such systems, we typically assume an ideal model. Such model-based control approaches cannot effectively facilitate optimal solutions with performance guarantees due to the discrepancy between the model and the actual cyber-physical system. On the other hand, in most CPS there is a large volume of data of a dynamic nature, which is added to the system gradually in real time and not altogether in advance. Thus, traditional supervised learning approaches cannot always facilitate robust solutions using data derived offline. By contrast, applying reinforcement learning approaches directly to the actual CPS might impose significant implications on the safety and robust operation of the system. In this talk, Malikopoulos discusses the challenges of supervised learning and model-based control approaches in several CPS applications, including self-learning powertrain control, power management control of hybrid electric vehicles, and optimal coordination of connected and automated vehicles. Then, Malikopoulos presents a theoretical framework founded at the intersection of control theory and learning that circumvents these challenges in deriving optimal strategies for CPS. In this framework, Malikopoulos and his research team aim to identify a sufficient information state for the CPS that takes values in a time-invariant space and use this information state to derive separated control strategies. Separated control strategies are related to the concept of separation between the estimation of the information state and control of the system. By establishing separated control strategies, they can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or the complexity of the system, and then use learning methods to learn the information state online while data are added gradually to the system in real time. This approach could effectively facilitate optimal solutions with performance guarantees in a wide range of CPS applications, such as emerging mobility systems, networked control systems, smart power grids, cooperative cyber-physical networks, cooperation of robots, and the Internet of Things.
Speaker
Andreas Malikopoulos is a Professor in the School of Civil & Environmental Engineering and the Director of the Information and Decision Science Lab at Cornell University. Prior to these appointments, he was the Terri Connor Kelly and John Kelly Career Development Professor in the Department of Mechanical Engineering (2017-2023) and the founding Director of the Sociotechnical Systems Center (2019-2023) at the University of Delaware (UD). Before he joined UD, he was the Alvin M. Weinberg Fellow (2010-2017) in the Energy & Transportation Science Division at Oak Ridge National Laboratory (ORNL), the Deputy Director of the Urban Dynamics Institute (2014-2017) at ORNL, and a Senior Researcher in General Motors Global Research & Development (2008-2010). He received a Diploma from the National Technical University of Athens, Greece, and his MS and Ph.D. degrees from the University of Michigan, Ann Arbor, in 2004 and 2008, respectively, all in Mechanical Engineering. His research interests span several fields, including analysis, optimization, and control of cyber-physical systems; decentralized stochastic systems; stochastic scheduling and resource allocation; and learning in complex systems. Dr. Malikopoulos is the recipient of several prizes and awards, including the 2007 Dare to Dream Opportunity Grant from the University of Michigan Ross School of Business, the 2007 University of Michigan Teaching Fellow, the 2010 Alvin M. Weinberg Fellowship, the 2019 IEEE Intelligent Transportation Systems Young Researcher Award, and the 2020 UD’s College of Engineering Outstanding Junior Faculty Award. He has been selected by the National Academy of Engineering to participate in the 2010 German-American Frontiers of Engineering (FOE) Symposium and organize a session on transportation at the 2016 European-American FOE Symposium. He has also been selected as a 2012 Kavli Frontiers of Science Scholar by the National Academy of Sciences. Dr. Malikopoulos has been an Associate Editor of the IEEE Transactions on Intelligent Vehicles and IEEE Transactions on Intelligent Transportation Systems from 2017 through 2020. He is an Associate Editor of Automatica and IEEE Transactions on Automatic Control, and a Senior Editor of IEEE Transactions on Intelligent Transportation Systems. He is a Senior Member of the IEEE, a Fellow of the ASME, and a member of the Board of Governors of the IEEE Intelligent Transportation Systems Society.
