April 28th
14:30 (Shannon amphitheatre, building 660) (see location):Jascha Sohl-dickstein (Google Brain)
Title: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Abstract:
I will present a method for building probabilistic models of complexdata-sets which is highly flexible, yet for which learning, sampling,
inference, and evaluation are still analytically or computationally
tractable. The essential idea, inspired by non-equilibrium statistical
physics, is to systematically and slowly destroy the structure in a data
distribution through an iterative forward diffusion process. We then train
a reverse diffusion process to restore that structure, yielding a
generative model of the data. By using a deep network to define the
diffusion process, we are able to rapidly learn, sample from, and evaluate
probabilities in deep generative models with thousands of layers or time
steps. I will demonstrate the effectiveness of this model on several
natural image datasets. I will also briefly discuss several other projects
applying ideas from nonequilibrium statistical mechanics to training,
sampling from, and predicting properties of machine learning models.
Publications:
- Sohl-Dickstein J, Weiss EA, Maheswaranathan N, Ganguli S. Deepunsupervised learning using nonequilibrium thermodynamics. International
Conference on Machine Learning. 2015. http://arxiv.org/abs/1503.03585
- Sohl-Dickstein J, Battaglino PB, DeWeese MR. New method for parameter
estimation in probabilistic models: minimum probability flow. Physical
review letters. 2011. https://journals.aps.org/prl/
abstract/10.1103/PhysRevLett.107.220601
- Sohl-Dickstein J, Mudigonda M, DeWeese MR. Hamiltonian Monte Carlo
without detailed balance. International Conference on Machine Learning.
2014. https://arxiv.org/abs/1409.5191
- Schoenholz SS, Gilmer J, Ganguli S, Sohl-Dickstein J. Deep Information
Propagation. International Conference on Learning Representations. 2017.
https://openreview.net/forum?id=H1W1UN9gg¬eId=H1W1UN9gg
Contact: guillaume.charpiat at inria.fr