Machine Learning Seminar Series with Yongxin Chen (Georgia Institute of Technology)

Fast Sampling of Diffusion Models

Diffusion models are a class of generative models that have led to the recent revolution of AI content generation, chief among which is the text-to-image application (e.g., DALLE2, Imagen, Stable diffusion, etc). Compared with other generative modeling techniques such as GANs, diffusion models achieve the best performance in terms of sample/image quality. However, the time consumption to generate a sample is considerably higher (typically several orders of magnitude more expensive than GANs). Diffusion models are built on the key idea of bridging a simple (Gaussian) distribution and a target distribution with a proper diffusion process modeled by a stochastic differential equation, and one needs to solve this stochastic differential equation through discretization to generate a new sample, incurring a high computational cost. In this talk, I will present three methods we proposed to accelerate the sampling of a diffusion model. The first method unifies the diffusion model and the normalizing flow, termed diffusion normalizing flow (DiffFlow), for generative modeling, by making the predefined forward process in diffusion models trainable. It is closely related to the Schrodinger bridge problem. In the second method, we develop an efficient algorithm to solve the learned backward process by leveraging certain structures of it. The resulting algorithm, termed diffusion exponential integrator sampler (DEIS), is currently the most efficient sampling algorithm for diffusion models (DEIS can generate high quality samples within 10 NFEs). In the last method, we consider the task of large content generation where the training data is limited. Our method, termed DiffCollage, makes it possible to efficiently generate large content using diffusion models trained on generating pieces of the large content.

Yongxin Chen received his BSc from Shanghai Jiao Tong University in 2011 and Ph.D. from University of Minnesota in 2016, both in Mechanical Engineering. He is currently an Assistant Professor in the School of Aerospace Engineering at Georgia Institute of Technology. He received the George S. Axelby Best Paper Award in 2017, the NSF Faculty Early Career Development Program (CAREER) Award in 2020, the A.V. `Bal' Balakrishnan Award in 2021, and the Donald P. Eckman Award in 2022. His current research interests are in the areas of control theory, machine learning, optimization, and robotics. He enjoys developing new algorithms and theoretical frameworks for real world applications.