from https://www.explainxkcd.com/wiki/images/c/c5/umwelt_counting_cards_harvard.png
About the seminar
In this seminar, we study different methods that can be applied to the problem of finding a good strategy to play Blackjack. Since the machine does not know the rules of Blackjack, it adopts (and modifies) random strategies. The data for learning will be the games that have been played. Some parts of the seminar will be devoted to implementing these methods in python.
Topics include the problem of reinforcement learning, inverse probability weighting and its relation to causality, Q-learning, contextual multi-armed bandits and the optimal strategy of playing BlackJack.
Seminar for Statistics: Learning Blackjack
Spring Semester, Mondays, 15:00pm to 17:00pm
(first lecture on February 22nd)
in room HG G 26.5
entry
in the Vorlesungsverzeichnis
We are looking forward to seeing you there.
Schedule
The current schedule including the distribution of talks can be found here.
Slides and Handouts
- Week 1 (Python and Blackjack): blackjack introduction, OOP
- Week 2 (Optimal Strategy to play Blackjack): UML diagram, Optimal Player, Strategic Player
- Week 3 (Multi-Armed Bandits): Multi-Armed Bandits
- Week 4 (Statistics of Multi-Armed Bandits): slides and handout
- Week 5 (Coding Session)
- Week 6 (Dynamic Programming): slides and handout
- Week 7 (Monte Carlo): slides and handout
- Week 8 (Temporal Difference): slides
- Week 9 (Causality): slides
- Week 10 (Inverse Probability Weighting): slides
Code repository
The blackjack framework we will be working with can be found in this repository.
Coding task I
The description of the first coding task can be found here. It is due on March 20th 23:59pm.
Coding task II
You may form a team with up to three people and you are supposed to hand in a solution (different from the explore-exploit assignment earlier). Please hand in a solution until May 25th 23:59pm.
Install and get to know python
We will be working with ipython notebook (now part of jupyter) and we therefore ask you to install python >= 2.7 (but not python 3 - there is a somewhat unfortunate parallel development - we will use python 2) and jupyter. I just followed the instructions on installation details using ANACONDA and everything worked fine. Please note that we really need you to have the installation ready but we cannot help you with any installation issues. In case you encounter problems, please ask friends and/or the internet. This might take some time! Also, please have a look at lecture 1 on this website. Those of you who can program/know python will only need a few minutes to scroll through. For all others, it will be a good introduction.
Literature
- Roger R. Baldwin et al.: The Optimum Strategy in Blackjack
- Richard S. Sutton and Andrew G. Barto: Reinforcement Learning: An Introduction
- Sebastien Bubeck and Nicolo Cesa-Bianchi: Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Contacts
Jonas Peters
(Seminar for Statistics, ETH Zurich and
MPI for Intelligent Systems, Tuebingen)
Christina Heinze (Seminar for Statistics, ETH Zurich)
Gian Thanei (Seminar for Statistics, ETH Zurich)
Nicolai Meinshausen (Seminar for Statistics, ETH Zurich)