Out-of-sample prediction in SEM: A simulation-based comparison of maximum likelihood and (consistent) partial least squares

Abstract

Structural equation modeling (SEM) is widely used to study relationships among unobserved constructs. Over recent years, the interest in computing predictions with SEM has grown. Different prediction procedures have been proposed for this purpose, including MLpredict, which is based on maximum likelihood (ML) estimation, and PLS(c)predict, which is based on partial least squares (PLS). The out-of-sample predictive performances of these procedures have not yet been compared. In our work, we conduct a comprehensive Monte Carlo study to compare out-of-sample predictions of MLpredict and PLS(c)predict for indicators associated with endogenous constructs, so-called operative predictions. Here, we vary key design factors regarding, for instance, the strengths of structural relations, the training sample size, and the construct types in the data generating processes, which determine the presence or absence of construct misspecification. Across the investigated conditions, MLpredict and PLS(c)predict exhibit similar predictive performance according to two assessment criteria, indicating that neither procedure is uniformly superior for operative prediction. Out-of-sample predictive performance is driven mainly by the strength of the relations between the variables, whereas training sample size and construct misspecification have comparatively little influence on it. An exception is the occasionally inflated prediction error of PLScpredict, which is in line with previous findings for ordinal indicators. Overall, our results suggest that, in the studied setting, fine-grained methodological differences between traditional SEM and PLS-based procedures matter less for out-of-sample predictive performance than properties of the data generating process, such as the coefficient of determination of endogenous constructs.