Scalable Variable Selection and Model Averaging for Latent Regression Models Using Approximate Variational Bayes

Abstract

We propose a fast and theoretically grounded method for Bayesian variable selection and model averaging in latent variable regression models. Our framework addresses three interrelated challenges: (i) intractable marginal likelihoods, (ii) exponentially large model spaces, and (iii) computational costs in large samples. We introduce a novel integrated likelihood approximation based on mean-field variational posterior approximations and establish its asymptotic model selection consistency under broad conditions. To reduce the computational burden, we develop an approximate variational Bayes scheme that fixes the latent regression outcomes for all models at initial estimates obtained under a baseline null model. Despite its simplicity, this approach locally and asymptotically preserves the model-selection behavior of the full variational Bayes approach to first order, at a fraction of the computational cost. Extensive numerical studies - covering probit, tobit, semi-parametric count data models and Poisson log-normal regression - demonstrate accurate inference and large speedups, often reducing runtime from days to hours with comparable accuracy. Applications to real-world data further highlight the practical benefits of the methods for Bayesian inference in large samples and under model uncertainty.