MnRI Colloquium: Ju Sun

Assistant Professor, Department of Computer Science & Engineering

Title: Deep Image Prior (and Its Cousin) for Inverse Problems: the Untold Stories.

Abstract: Deep image prior (DIP) parametrizes visual objects as outputs of deep neural networks (DNNs); its cousin neural implicit representation (NIR) directly parametrizes visual objects as DNNs. These stunningly simple ideas, when integrated into natural optimization formulations for visual inverse problems, have matched or even beaten the state-of-the-art methods on numerous visual reconstruction tasks, although not driven by massive amounts of training data. Despite the remarkable successes, the over parametrized DNNs used are
typically powerful enough to also fit the noise besides the desired visual contents (i.e., overfitting), and the fitting process can take up to tens of minutes on sophisticated GPU cards to converge to a reasonable solution. In this talk, I’ll describe our recent efforts to combat these practicality issues around DIP and NIR, and how careful calibration of DIP models (or variants) can help to solve challenging visual reconstruction problems, such as blind image deblurring and phase retrieval, in unprecedented regimes.

Joint work with Taihui Li, Hengkang Wang, Zhong Zhuang, Hengyue Liang, Le Peng, and Tiancong Chen.

Related papers:
Early Stopping for Deep Image Prior  https://arxiv.org/abs/2112.06074
Self-Validation: Early Stopping for Single-Instance Deep Generative Priors https://arxiv.org/abs/2110.12271
Blind Image Deblurring with Unknown Kernel Size and Substantial Noise https://arxiv.org/abs/2208.09483