[Statlist] Reminder: FDS Virtual Seminar by Matus Telgarsky, University of Illinois Urbana-​Champaign, 27.11.2020

[Maurer Letizia]

We are glad to announce the following online talk in the ETH Foundations of Data Science Seminar 

"The dual of the margin: improved analyses and rates of gradient descent's implicit bias"   
by Matus Telgarsky, University of Illinois Urbana-​Champaign

Time: Friday, 27.11.2020 at 16.00 h
Place: Airmeet lecture, https://www.airmeet.com/e/11670120-26a3-11eb-928f-8f3ac926165c

Abstract: The implicit bias of gradient descent has arisen as a promising explanation for the good generalization properties of deep networks (Soudry-​Hoffer-Nacson-Gunasekar-Srebro, 2018). The purpose of this talk is to demonstrate the effectiveness of a certain dual problem in the analysis of this implicit bias. Concretely, this talk will develop this dual, as well as a variety of consequences in linear and nonlinear settings. In the linear case, this dual perspective firstly will allow a characterization of the implicit bias, even outside the standard setting of exponentially-​tailed losses; in this sense, it is gradient descent, and not a particular loss structure which leads to implicit bias. Secondly, invoking duality in the margin convergence analysis will yield a fast 1/t rate; by contrast, all prior analyses never surpassed 1/sqrt{t}, even in the well-​studied boosting setting. In the nonlinear case, duality will enable the proof of a gradient alignment property: asymptotically, the parameters and their gradients become colinear. Although abstract, this property in turn implies various existing and new margin maximization results. Joint work with Ziwei Ji. bio: Matus Telgarsky is an assistant professor at the University of Illinois, Urbana-​Champaign, specializing in deep learning theory. He was fortunate to receive a PhD at UCSD under Sanjoy Dasgupta. Other highlights include: co-​founding, in 2017, the Midwest ML Symposium (MMLS) with Po-​Ling Loh; receiving a 2018 NSF CAREER award; organizing a Simons Insititute summer 2019 program on deep learning with Samy Bengio, Aleskander Madry, and Elchanan Mossel. In 2020, meanwhile, he's thankful to be alive and employed.

Organisers: A. Bandeira, H. Bölcskei, P. Bühlmann, J. Buhmann, N. He, T. Hofmann, A. Krause, R. Kyng, A. Lapidoth, H.-A. Loeliger, M. Maathuis, N. Meinshausen, S. Mishra, G. Rätsch, Ch. Schwab, D. Steurer, S. van de Geer, F. Yang, R. Zenklusen

Seminar website: https://math.ethz.ch/sfs/news-and-events/data-science-seminar.html