Student Seminar in Statistics:
Inference in Some Non-Standard Regression Problems

General information

Course content

Abstract

Review of some non-standard regression models and the statistical properties of estimation methods in such models.

Linear regression is one of the most used models for prediction and hence one of the most understood in statistical literature. However, linearity might be too simplistic to capture the actual relationship between some response and given covariates. Also, there are many real data problems where linearity is plausible but the actual pairing between the observed covariates and responses is completely or partially lost. In this seminar, we review some of the non-classical regression models and the statistical properties of the estimation methods considered by well-known statisticians and machine learners. This will encompass:

1. Monotone regression
2. Single index model
3. Unlinked regression

Objective

The main goal is for the students to discover some less known regression models which either generalize the well-known linear model (for example monotone regression) or violate some of the most fundamental assumptions (as in shuffled or unlinked regression models).

Literature In the following is the tentative material that will be studied and presented by each pair of students. Some of the items might change - the updated literature list and links to the material can be found in the Moodle course

1. Chapter 2 from the book “Nonparametric estimation under shape constraints” by P. Groeneboom and G. Jongbloed, 2014, Cambridge University Press
2. “Estimating a convex function in nonparametric regression” by M. Birke and H. Dette, 2007, Scandinavian Journal of Statistics, Volume 34, 384-404
3. “Nonparametric shape-restricted regression” by A. Guntuoyina and B. Sen, 2018, Statistical Science, Volume 33, 568-594
4. “Approximation by log-concave distributions, with applications to regression” by L. D ̈umbgen, R. Samworth and D. Schuhmacher, 2011, Annals of Statistics, Volume 39, 702-730
5. “Feedforward Networks with Monotone Constraints” by H. Zhang and Z. Zhang, IJCNN’99. International Joint Conference on Neural Networks, Volume 3, 1999
6. “Least squares estimation in the monotone single index model” by F. Balabdaoui, C. Durot and H. K. Jankowski, Journal of Bernoulli, 2019, Volume 4B, 3276-3310
7. Chapter 1 (Introduction) from the book “High-Dimensional Statistics : A Non-Asymptotic Point of View” by M. J. Wainwright, Cambridge University Press
8. “Sharp thresholds for high dimensional and noisy sparsity recovery using `1-constrained quadratic programming (Lasso)” by M. J. Wainwright, 2009, IEEE transactions in Information Theory, Volume 55, 1-19
9. “Denoising linear models with permuted data” by A. Pananjady, M. Wainwright and T. A. Courtade, 2017, IEEE International Symposium on Information Theory, 446-450
10. “Unlinked monotone regression” by F. Balabdaoui, C. Doss and C. Durot, 2021, JMLR, Volume 22, 1-60
11. “Linear regression with unmatched data : A deconvolution prespective” by M. Azadkia and F. Balabdaoui
12. “Uncoupled isotonic regression via minimum Wasserstein deconvolution” by P. Rigollet and J. Weed, 2019, Information and Inference, Volume 00, 1-27

Please see the Moodle course for more details.

Notice

This website is only to give an overview of the course and what is offered. Communication and organization is done through the Moodle course. Please contact the assistants if you do not have access after registering for the course.