Federated Reinforcement Learning: Statistical, Communication and Computation Trade-offs

Data Science Seminar

Yuejie Chi
Carnegie Mellon

Abstract

Reinforcement learning (RL), concerning decision making in uncertain environments, lies at the heart of modern artificial intelligence. Due to the high dimensionality, training of RL agents typically requires a significant amount of computation and data to achieve desirable performance. However, data collection can be extremely time-consuming with limited access in real-world applications, especially when performed by a single agent. On the other hand, it is plausible to leverage multiple agents to collect data simultaneously, under the premise that they can learn a global policy collaboratively without the need of sharing local data in a federated manner. This talk addresses the fundamental statistical, communication and computation trade-offs in the algorithmic designs of federated RL algorithms, covering both blessings and curses in the presence of data and task heterogeneities across the agents.