September, Tuesday 19th

11:00 (Shannon amphitheatre, building 660) (see location):

Carlo Lucibello

(Politecnico di Torino)

Title: Probing the energy landscape of Artificial Neural Networks

Abstract:

Training neural networks with discrete synapses has long been considered a challenging task even for the simplest neural architectures.
In this talk I'll present a series of results which emerged from a large-deviation analysis using tools from Statistical Physics, which show that the learning problem can be made algorithmically very simple by maximizing a "local entropy": explicitly seeking extensive regions in the space of configurations with low energy. Such regions also have some highly desirable properties, in particular very good generalization capabilities.
A class of general optimization algorithms is presented, along with numerical results in shallow and deep learning assessing their effectiveness.

Bibliography:

C. Baldassi, C. Borgs, J. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina. "Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes", PNAS (2016)




Contact: guillaume.charpiat at inria.fr