Improved Probabilistic Principal Component Analysis for Application to Reduced Order Modeling

Abstract

In our previous work, a reduced order model (ROM) for a stochastic system was made, where noisy data was projected onto principal component analysis (PCA)-derived basis vectors to obtain an accurate reconstruction of the noise-free data. That work used techniques designed for deterministic data, PCA was used for the basis function generation and $L_2$ projection was used to create the reconstructions. In this work, probabilistic approaches are used. The probabilistic PCA (PPCA) is used to generate the basis, which then allows the noise in the training data to be estimated. PPCA has also been improved so that the derived basis vectors are orthonormal and the variance of the basis expansion coefficients over the training data set can be estimated. The standard approach assumes a unit variance for these coefficients. Based on the results of the PPCA, model selection criteria are applied to automatically choose the dimension of the ROM. In our previous work, a heuristic approach was used to pick the dimension. Lastly, a new statistical approach is used for the projection step where the variance information obtained from the improved PPCA is used as a prior to improve the projection. This gives improved accuracy over $L_2$ projection when the projected data is noisy. In addition, the noise statistics for the projected data are not assumed to be the same as that of the training data, but are estimated in the projection process. The entire approach gives a fully stochastic method for computing a ROM from noisy training data, determining ideal model selection, and projecting noisy test data, thus enabling accurate predictions of noise-free data from data that is dominated by noise.