Student Seminar in Statistics:
Inference in Some Non-Standard Regression Problems

General information

Course content

Review of some non-standard regression models and the statistical properties of estimation methods in such models.

The main goal is the students get to discover some less known regression models which either generalize the well-known linear model (for example monotone regression) or violate some of the most fundamental assumptions (as in shuffled or unlinked regression models).

Linear regression is one of the most used models for prediction and hence one of the most understood in statistical literature. However, linearity might be too simplistic to capture the actual relationship between some response and given covariates. Also, there are many real data problems where linearity is plausible but the actual pairing between the observed covariates and responses is completely lost or at partially. In this seminar, we review some of the non-classical regression models and the statistical properties of the estimation methods considered by well-known statisticians and machine learners. This will encompass:

1. Monotone regression
2. Single index model
3. Unlinked regression

In the following is the tentative material that will be read and studied by each pair of students (all the items listed below are available through the ETH electronic library or arXiv). Some of the items might change.

1. Chapter 2 from the book "Nonparametric estimation under shape constraints" by P. Groeneboom and G. Jongbloed, 2014, Cambridge University Press
2. "Estimating a Convex Function in Nonparametric Regression" by M. Birke and H. Dette, 2007, Scandinavian Journal of Statistics, Volume 34, 384-404
3. "Nonparametric shape-restricted regression" by A. Guntuoyina and B. Sen, 2018, Statistical Science, Volume 33, 568-594
4. "Least squares estimation in the monotone single index model" by F. Balabdaoui, C. Durot and H. K. Jankowski, Journal of Bernoulli, 2019, Volume 4B, 3276-3310
5. "Semiparametric Efficiency in Convexity Constrained Single-Index Model" by A. K. Kuchibhotla, R. K. Patra and B. Sen, 2021, JASA (Theory and Methods), 1-15
6. "Sharp thresholds for high dimensional and noisy sparsity recovery using l1-constrained quadratic programming (Lasso)" by M. Wainwright, 2009, IEEE transactions in Information Theory, Volume 55, 1-19
7. "The Isotron Algorithm: High-Dimensional Isotonic Regression" by A. T. Kalai and R. Sastry, 2009, COLT
8. "Forward selection and estimation in high dimensional single index models", by S. Luo and S. Ghosal, 2016, Statistical Methodology, Volume 33, 172-179
9. "Denoising linear models with permuted data" by A. Pananjady, M. Wainwright and T. A. Courtade and , 2017, IEEE International Symposium on Information Theory, 446-450
10. "Unlinked Monotone Regression" by Fadoua Balabdaoui, Charles Doss and Cecile Durot, 2021, JMLR, Volume 22, 1-60
11. "Uncoupled isotonic regression via minimum Wasserstein deconvolution" by P. Rigollet and J. Weed, 2019, Information and Inference, Volume 00, 1-27

The students need to be comfortable with regression models, classical estimation methods (Least squares, Maximum Likelihood estimation...), rates of convergence, asymptotic normality, etc.