A robust and stable hybrid neural network/finite element method for 2D flows that generalizes to different geometries

Abstract

The deep neural network multigrid solver (DNN-MG) combines a coarse-grid finite element simulation with a deep neural network that corrects the solution on finer grid levels, thereby improving the computational efficiency. In this work, we discuss various design choices for the DNN-MG method and demonstrate significant improvements in accuracy and generalizability when applied to the solution of the nonstationary Navier-Stokes equations. We investigate the stability of the hybrid simulation and show how the neural networks can be made more robust with the help of replay buffers. After an initial single-step training, we run the hybrid simulation for extended periods and compute new reference solutions of the neural network perturbed state for each step. By retraining on this data, the error caused by the neural network over multiple time-steps due to distributional shift can be effectively reduced without the need for a differentiable numerical solver. Furthermore, we compare multiple neural network architectures, including recurrent neural networks and Transformers, and study their ability to utilize more information from an increased temporal and spatial receptive field. Transformers allow us to make use of information from cells outside the predicted patch even with unstructured meshes while maintaining the locality of our approach. This can further improve the accuracy of DNN-MG without a significant impact on performance.