7 - Regularized Regression Methods

Summary

Chapter 7 is dedicated to regularized regression methods, which – by penalizing models that are too complex – are capable of providing a reasonable tradeoff between bias and variance. Ridge regression implements L₂ regularization, which results in more generalizable models, but does not perform any feature selection. L₁ penalty used by the lasso allows, however, for simultaneous regularization and feature selection. The elastic net algorithm combines the two approaches by applying both L₁ and L₂ penalties, which allows for solutions combining the advantages of both ridge regression and the lasso. The chapter concludes by discussing a general class of L_q-regularized least squares optimization problems.