Scalable solution to crossed random effects model with random slopes

Abstract

The crossed random effects model is widely used, finding applications in various fields such as longitudinal studies, e-commerce, and recommender systems, among others. However, these models encounter scalability challenges, as the computational time for standard algorithms grows superlinearly with the number N of observations in the data set, commonly $\Omega(N^{3/2})$ or worse. Recent published works present scalable methods for crossed random effects in linear models and some generalized linear models, but those methods only allow for random intercepts. In this paper, we devise scalable algorithms for models that include random slopes. This addition brings substantial difficulty in estimating the random-effect covariance matrices in a scalable way. We address this issue by using a variational EM algorithm. Our proposed approach accommodates both diagonal covariance matrices and cases where no structure is assumed-a scenario common in fields such as psychology and neuroscience. In simulations, the proposed method is substantially faster than standard methods for large $N$. It is also more efficient than ordinary least squares which has a problem of greatly underestimating the sampling uncertainty in parameter estimates. We illustrate the new method on a MovieLens dataset, as well as a large data set (five million observations) from the online retailer Stitch Fix.