Mathematical Tools in Machine Learning

Autumn semester 2019

General information

Lecturer Fadoua Balabdaoui
Assistants Loris Michel
Lectures Thu 10-12 HG E 5
Course catalogue data >>

Course content

In the exploding world of artifical intelligence and automated learning, there is an urgent need to go back to the basis of what is driving many of the well-establsihed methods in statistical learning. The students attending the lectures will get acquainted with the main theoretical results needed to establish the theory of statistical learning. We start with defining what is meant by learning a task, a training sample, the trade-off between choosing a big class of functions (hypotheses) to learn the task and the difficulty of estimating the unknown function (generating the observed sample). The course will also cover the notion of learnability and the conditions under which it is possible to learn a task. In a second part, the lectures will cover algoritmic apsects where some well-known algorithms will be described and their convergence proved. Through the exerices classes, the students will deepen their understanding using their knowledge of the learned theory on some new situations, examples or some counterexamples.

Announcements

  • For phd students who want to get credits for this course, the requirement is to submit solutions of 3 problem sets out of 4
  • Question hours before exam: 21.01.20: room HG G 19.1 at 16:00, 30.01.20: room LEE C 104 at 16:00

Course materials

Week Topic
Week 1
  • Slides
  • Reading: Chapters 1-3
Week 2
  • Lecture on the blackboard
  • Reading: Chapters 3 and 4
Week 3 Exercise session 1
  • Exercise 2.3 (questions 1, 2 and 3)
  • Exercise 3.1, 3.2 (question 2), 3.3, 3.6 and 3.7
  • Exercise 4.1
Solutions
Week 4
  • Lecture on the blackboard
  • Reading: Chapters 4 and 9
Week 5
  • Slides
  • Reading: Chapter 9 and Chapter 5
Week 6 Exercise session 2
  • Exercise 9.1, 9.3, 9.5
  • Exercise 5.1
  • Exercise 5.3 (optional)
Solutions
Week 7
  • Lecture on the blackboard
  • Reading: Chapters 5 and 6
Week 8
  • Lecture on the blackboard
  • Reading: Chapter 6
Week 9 Exercise session 3
  • Reproduce the proof of Sauer's lemma (p. 49-50)
  • Exercises 6.1, 6.4, 6.8
  • Exercise 6.9 (optional)
Solutions
Week 10
  • Slides
  • Reading: Chapters 6, 11 and 12
Week 11
  • Slides
  • Reading: Chapter 12 and Chapter 14 (14.1-14.3)
Week 12
  • Lecture on the blackboard
  • Reading: Chapter 14 (14.3, 14.5.1, 14.5.2, 14.5.3)
Week 13 Exercise session 4
  • Exercise 11.1
  • Exercise 12.2 and 12.4 (optional)
  • Exercise 14.3
  • Exercise 13.1 (1st question)
Solutions

Exercise classes

Series

Exercises Solutions
Series 1 Solutions 1 TBA

Literature


    Main reference:
    Shalev-Shwartz, S. and Ben-David, S., 2014. Understanding machine learning: From theory to algorithms. Cambridge university press

    Additional references:
  • Anthony, M. and Bartlett, P.L., 2009. Neural network learning: Theoretical foundations. Cambridge university press
  • Lecture Notes of Philippe Rigollet On Mathematics of Machine Learning