Robustified L2Boosting

Roman Werner Lutz and Peter Bühlmann

December 2006

Abstract:
We consider five robustifications of L₂Boosting for linear regression with various robustness properties. The first two use the Huber loss as implementing loss function for boosting and the second two use robust simple linear regression for the fitting in L₂Boosting (i.e. robust base learners). Both concepts can be applied with or without down-weighting of leverage points. Our last method uses robust correlation estimates and appears to be most robust. Crucial advantages of all methods are that they don't compute covariance matrices of all covariates and that they don't have to identify multivariate leverage points. When there are no outliers, the robust methods are only slightly worse than L₂Boosting. In the contaminated case though, the robust methods outperform L₂Boosting by a large margin. Some of the robustifications are also computationally highly efficient and therefore well suited for truly high dimensional problems.

Download: Compressed Postscript (140 Kb) PDF (187 Kb).