Review of some non-standard regression models and the statistical properties of estimation methods in such models.Objective
The main goal is the students get to discover some less known regression models which either generalize the well-known linear model (for example monotone regression) or violate some of the most fundamental assumptions (as in shuffled or unlinked regression models).Content
Linear regression is one of the most used models for prediction and hence one of the most understood in statistical literature. However, linearity might be too simplistic to capture the actual relationship between some response and given covariates. Also, there are many real data problems where linearity is plausible but the actual pairing between the observed covariates and responses is completely lost or at partially. In this seminar, we review some of the non-classical regression models and the statistical properties of the estimation methods considered by well-known statisticians and machine learners. This will encompass:
- Monotone regression
- Single index model
- Unlinked regression
In the following is the tentative material that will be read and studied by each pair of students (all the items listed below are available through the ETH electronic library or arXiv). Some of the items might change.
- Chapter 2 from the book "Nonparametric estimation under shape constraints" by P. Groeneboom and G. Jongbloed, 2014, Cambridge University Press
- "Estimating a Convex Function in Nonparametric Regression" by M. Birke and H. Dette, 2007, Scandinavian Journal of Statistics, Volume 34, 384-404
- "Nonparametric shape-restricted regression" by A. Guntuoyina and B. Sen, 2018, Statistical Science, Volume 33, 568-594
- "Least squares estimation in the monotone single index model" by F. Balabdaoui, C. Durot and H. K. Jankowski, Journal of Bernoulli, 2019, Volume 4B, 3276-3310
- "Semiparametric Efficiency in Convexity Constrained Single-Index Model" by A. K. Kuchibhotla, R. K. Patra and B. Sen, 2021, JASA (Theory and Methods), 1-15
- "Sharp thresholds for high dimensional and noisy sparsity recovery using l1-constrained quadratic programming (Lasso)" by M. Wainwright, 2009, IEEE transactions in Information Theory, Volume 55, 1-19
- "The Isotron Algorithm: High-Dimensional Isotonic Regression" by A. T. Kalai and R. Sastry, 2009, COLT
- "Forward selection and estimation in high dimensional single index models", by S. Luo and S. Ghosal, 2016, Statistical Methodology, Volume 33, 172-179
- "Denoising linear models with permuted data" by A. Pananjady, M. Wainwright and T. A. Courtade and , 2017, IEEE International Symposium on Information Theory, 446-450
- "Unlinked Monotone Regression" by Fadoua Balabdaoui, Charles Doss and Cecile Durot, 2021, JMLR, Volume 22, 1-60
- "Uncoupled isotonic regression via minimum Wasserstein deconvolution" by P. Rigollet and J. Weed, 2019, Information and Inference, Volume 00, 1-27
The students need to be comfortable with regression models, classical estimation methods (Least squares, Maximum Likelihood estimation...), rates of convergence, asymptotic normality, etc.
Welcome to the website of the course Student Seminar in Statistics: Inference in Non-Classical Regression Models!
The first class will be an introductory lecture and will take place on Monday, 26.09.2022.
We are looking forward to seeing you!
Assignment of topics:
The topics will be assigned via Doodle, which we will send out at 19.09.2022. The topics will be assigned based on the "first come first serve" rule.
The first student presentation will take place on 03.10.2022.
Please send an email to firstname.lastname@example.org ASAP in case you decide to not take part in the seminar.
Course material and schedule
The registered students will be divided into pairs to work on the papers. Everyone is expected to participate actively during all lectures. Questions and discussions are strongly encouraged in this class!
The presentations should last roughly 2 x 25 minutes, with a 5-10 minute break in between. One of the assistants will meet with you before your presentation, to answer questions about the material and to give feedback on your planned presentation. Please also see the FAQ for further details.
Please connect to the ETH VPN if you can not access the following papers.
|Week 1 (26.09.2022)||Introductory Lecture
|Week 2 (03.10.2022)||Group 1: Basic Estimation Problems with Monotonicity Constraints
|Week 3 (10.10.2022)|| Group 2: Estimating a Convex Function in Nonparametric Regression
|Week 4 (17.10.2022)|| Group 3: Nonparametric Shape-Restricted Regression
|Week 5 (24.10.2022)||Group 4: Least Squares Estimation in the Monotone Single Index Model
|Week 6 (31.10.2022)||Group 5: Semiparametric Efficiency in Convexity Constrained Single-Index Model
|Week 7 (07.11.2022)||Group 6: Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using ℓ1 -Constrained Quadratic Programming (Lasso)
|Week 8 (14.11.2022)||Group 7: The Isotron Algorithm: High-Dimensional Isotonic Regression
|Week 9 (21.11.2022)||Group 8: Forward selection and estimation in high dimensional single index models
|Week 10 (28.11.2022)||Group 9: Denoising Linear Models with Permuted Data
|Week 11 (05.12.2022)||Group 10: Unlinked Monotone Regression
|Week 12 (12.12.2022)||Group 11: Uncoupled isotonic regression via minimum Wasserstein deconvolution
|Week 13 (19.12.2022)|| No class
- How long should the presentation be?
The total presentation time is 50 minutes. Each student should present roughly half of the time. We advise you to split the presentation in two parts of about 25 minutes each, with a 5-10 minute break in between. Please make sure to practice so that you don't go over your time! We highly encourage interaction and discussion with the audience, both during and after your talk. If this happens during your talk, this will not be counted as presentation time.
- Should I use a certain template for my slides?
You can use any template you like. We recommend using one of the ETH presentation templates.
- How should the presentation be structured?
The main purpose of the presentation is to transmit knowledge to the audience. So, after reading the material, please take a step back and try to put yourself in the shoes of the audience: What do they already know? What would they find most interesting? What would be helpful examples? We will also provide further guidelines for the presentations during the first lecture.
- Do I need to bring my own laptop to present my slides?
Ideally, yes. If you do not have a laptop, or you do not have a way of connecting to the projector, please let the assistants know in advance.
- Will my slides be published somewhere?
Yes, all slides will be published on the course website after the presentation. Please make sure to respect copyright. In particular, if you include any images or tables not created by yourself in the presentation, make sure to include the source of the image/table as well.
- What is the role of the assistants?
The assistant in charge for your group gives you guidance and feedback prior to your presentation. You will have a chance to meet with the assistant before your presentation. The assistant will contact you about the details of the meeting.
- Do I have to attend all lectures?
Yes, attendance at all lectures is compulsory. If you have to miss a class (due to illness or some force-major), please contact Prof. Dr. Fadoua Balabdaoui directly.