[Statlist] Reminder: (Please notice the link update) FDS Virtual Seminar by Ivan Dokmanić, Universität Basel and University of Illinois at Urbana-Champaign, 19.11.2020

[Maurer Letizia]

We are glad to announce the following online talk in the ETH Foundations of Data Science Seminar 

"Injective Neural Networks for Inference and Inverse Problems"   
by Ivan Dokmanić, Universität Basel and University of Illinois at Urbana-Champaign

Time: Thursday, 19.11.2020 at 16.00 h
Place: zoom lecture, https://ethz.zoom.us/j/92746609100, Meeting-ID: 927 4660 9100

Abstract: Injectivity plays an important role in generative models where it enables inference; in inverse problems and compressed sensing with generative priors it is a precursor to well posedness. We establish sharp characterizations of injectivity of fully-​connected and convolutional ReLU layers and networks. We begin by a layerwise analysis and show that an expansivity factor of two is necessary and sufficient for injectivity by constructing appropriate weight matrices. We show that global injectivity with iid Gaussian matrices, a commonly used tractable model, requires larger expansivity between 3.4 and 5.7. We also characterize the stability of inverting an injective network via worst-​case Lipschitz constants of the inverse. Next, we use arguments from differential topology to study injectivity of deep networks and prove that any Lipschitz map can be approximated by an injective ReLU network; we . Finally, using an argument based on random projections, we show that an end-​to-end---rather than layerwise-​--doubling of the dimension suffices for injectivity. We close with numerical experiments on injective generative models showing that injectivity improves inference.

Organisers: A. Bandeira, H. Bölcskei, P. Bühlmann, J. Buhmann, N. He, T. Hofmann, A. Krause, R. Kyng, A. Lapidoth, H.-A. Loeliger, M. Maathuis, N. Meinshausen, S. Mishra, G. Rätsch, Ch. Schwab, D. Steurer, S. van de Geer, F. Yang, R. Zenklusen

Seminar website: https://math.ethz.ch/sfs/news-and-events/data-science-seminar.html