Inference in latent factor regression with clusterable features

Abstract

Regression models, in which the observed features $X \in \R^p$ and the response $Y \in \R$ depend, jointly, on a lower dimensional, unobserved, latent vector $Z \in \R^K$, with $K< p$, are popular in a large array of applications, and mainly used for predicting a response from correlated features. In contrast, methodology and theory for inference on the regression coefficient $\beta$ relating $Y$ to $Z$ are scarce, since typically the un-observable factor $Z$ is hard to interpret. Furthermore, the determination of the asymptotic variance of an estimator of $\beta$ is a long-standing problem, with solutions known only in a few particular cases. To address some of these outstanding questions, we develop inferential tools for $\beta$ in a class of factor regression models in which the observed features are signed mixtures of the latent factors. The model specifications are practically desirable, in a large array of applications, render interpretability to the components of $Z$, and are sufficient for parameter identifiability. Without assuming that the number of latent factors $K$ or the structure of the mixture is known in advance, we construct computationally efficient estimators of $\beta$, along with estimators of other important model parameters. We benchmark the rate of convergence of $\beta$ by first establishing its $\ell_2$-norm minimax lower bound, and show that our proposed estimator is minimax-rate adaptive. Our main contribution is the provision of a unified analysis of the component-wise Gaussian asymptotic distribution of $\wh \beta$ and, especially, the derivation of a closed form expression of its asymptotic variance, together with consistent variance estimators. The resulting inferential tools can be used when both $K$ and $p$ are independent of the sample size $n$, and when both, or either, $p$ and $K$ vary with $n$, while allowing for $p > n$.