CSE DSI Machine Learning Seminar with Hongkai Zhao (Math, Duke University)

Numerical understanding of neural networks: from representation to learning dynamics

In this talk we present both numerical analysis and experiments to study a few basic computational issues in practice: (1) the numerical error one can achieve given a finite machine precision, (2) the learning dynamics and computation cost to achieve a given accuracy, and (3) stability with respect to perturbations. These issues are addressed for both approximation and optimization in asymptotic and non-asymptotic regimes.