November, Wednesday 22nd

14:30 (Room 2014, building 660) (see location):

Marylou Gabrié

(ENS Paris, Laboratoire de Physique Statistique)

Title: Mean-Field Framework for Unsupervised Learning with Boltzmann Machines

Abstract:

Boltzmann machines are undirected neural networks useful for unsupervised machine learning. In particular, a simple bipartite version - called Restricted Boltzmann machines (RBMs) - has been widely popularized by the discovery of fast training algorithms, relying on approximate Monte Carlo Markov Chains. Realizing that training RBMs is closely related to the inverse Ising problem, a notoriously hard statistical physics problem, we designed an alternative deterministic procedure based on the Thouless-Anderson-Palmer approach. Our algorithm, improving on the naive mean-field approximation, provides performance equal to the commonly used MCMC algorithms while also providing a clear and easy to evaluate objective function to follow progress along training. Moreover, this strategy can be generalized in many ways, including for new network architectures or for new types of data. Finally, a particularly exciting application of this new framework is the integration of learned priors in Bayesian inference problems.




Contact: guillaume.charpiat at inria.fr
All TAU seminars: here