Justin Sirignano - Laws of Large Numbers for Neural Networks

The asymptotics of neural networks can be studied as the number of hidden units become large. Two different limits can be established, depending upon the choice of normalization for the network. The "mean-field limit" satisfies a PDE while the "kernel limit" satisfies an ODE. The differences between these two types of limits and their relevance to deep learning applications will be discussed. The presentation will conclude with a brief overview of some of our current research projects on reinforcement learning, online stochastic optimization, deep learning methods for solving high-dimensional PDEs, and deep learning-based PDE models.