The Accessible Lasso Models

Abstract

A new line of research on the lasso exploits the beautiful geometric fact that the lasso fit is the residual from projecting the response vector $y$ onto a certain convex polytope. This geometric picture also allows an exact geometric description of the set of accessible lasso models for a given design matrix, that is, which configurations of the signs of the coefficients it is possible to realize with some choice of $y$. In particular, the accessible lasso models are those that correspond to a face of the convex hull of all the feature vectors together with their negations. This convex hull representation then permits the enumeration and bounding of the number of accessible lasso models, which in turn provides a direct proof of model selection inconsistency when the size of the true model is greater than half the number of observations.