April 28th

14:30 (Shannon amphitheatre, building 660) (see location):

Jascha Sohl-dickstein (Google Brain)

Title: Deep Unsupervised Learning using Nonequilibrium Thermodynamics


I will present a method for building probabilistic models of complex
data-sets which is highly flexible, yet for which learning, sampling,
inference, and evaluation are still analytically or computationally
tractable. The essential idea, inspired by non-equilibrium statistical
physics, is to systematically and slowly destroy the structure in a data
distribution through an iterative forward diffusion process. We then train
a reverse diffusion process to restore that structure, yielding a
generative model of the data. By using a deep network to define the
diffusion process, we are able to rapidly learn, sample from, and evaluate
probabilities in deep generative models with thousands of layers or time
steps. I will demonstrate the effectiveness of this model on several
natural image datasets. I will also briefly discuss several other projects
applying ideas from nonequilibrium statistical mechanics to training,
sampling from, and predicting properties of machine learning models.


- Sohl-Dickstein J, Weiss EA, Maheswaranathan N, Ganguli S. Deep
unsupervised learning using nonequilibrium thermodynamics. International
Conference on Machine Learning. 2015. http://arxiv.org/abs/1503.03585
- Sohl-Dickstein J, Battaglino PB, DeWeese MR. New method for parameter
estimation in probabilistic models: minimum probability flow. Physical
review letters. 2011. https://journals.aps.org/prl/
- Sohl-Dickstein J, Mudigonda M, DeWeese MR. Hamiltonian Monte Carlo
without detailed balance. International Conference on Machine Learning.
2014. https://arxiv.org/abs/1409.5191
- Schoenholz SS, Gilmer J, Ganguli S, Sohl-Dickstein J. Deep Information
Propagation. International Conference on Learning Representations. 2017.

Contact: guillaume.charpiat at inria.fr