On the efficiency of the de-biased Lasso

Abstract

We consider the high-dimensional linear regression model $Y = X β^0 + ε$ with Gaussian noise $ε$ and Gaussian random design $X$. We assume that $Σ:= E X^T X / n$ is non-singular and write its inverse as $Θ:= Σ^{-1}$. The parameter of interest is the first component $β_1^0$ of $β^0$. We show that in the high-dimensional case the asymptotic variance of a debiased Lasso estimator can be smaller than $Θ_{1,1}$. For some special such cases we establish asymptotic efficiency. The conditions include $β^0$ being sparse and the first column $Θ_1$ of $Θ$ being not sparse. These conditions depend on whether $Σ$ is known or not.