MnRI Colloquium: Andrew Lamperski

Associate Professor, Department of Electrical & Computer Eng.  

Title: New Quantitative Bounds on Recursive Stochastic Algorithms with Applications to Reinforcement Learning and System Identification.

Abstract: Recursive stochastic algorithms are pervasive in machine learning applications, system identification, and control. Examples include stochastic gradient descent, Q-learning, recursive least-squares, and Langevin algorithms. In reinforcement learning, control, and system identification applications, measurements have dependencies across time that make the quantitative analysis of the associated algorithms more challenging. In this talk, we describe how a wide variety of these algorithms can be analyzed in a unified framework known as L-mixing. The theory of L-mixing processes quantifies how statistical dependencies in a time series decay over time. Our first result will be to show how in scenarios, if the system generating the data is stable, then the data will be L-mixing. From here, we will show several algorithms from machine learning, and system identification can be cast into this L-mixing framework. Finally, we will show the L-mixing properties lead to clean, quantitative bounds on the convergence of these algorithms.