arXiv subjects

Arvind Mohan

Publications and source records attributed to Arvind Mohan.

12 recordsLinked to original sources

arXiv2025

Towards Reasoning for PDE Foundation Models: A Reward-Model-Driven Inference-Time-Scaling Algorithm

Partial Differential Equations (PDEs) are the bedrock for modern computational sciences and engineering, and inherently computationally expensive. While PDE foundation models have shown much promise for simulating such complex spatio-temporal phenomena, existing models remain constrained by the pretraining datasets and struggle with auto-regressive rollout performance, especially in out-of-distribution (OOD) cases. Furthermore, they have significant compute and training data requirements which hamper their use in many critical applications. Inspired by recent advances in ``thinking" strategies used in large language models (LLMs), we introduce the first test-time computing (TTC) strategy for PDEs that utilizes computational resources during inference to achieve more accurate predictions with fewer training samples and smaller models. We accomplish this with two types of reward models that evaluate predictions of a stochastic based model for spatio-temporal consistency. We demonstrate this method on compressible Euler-equation simulations from the PDEGym benchmark and show that TTC captures improved predictions relative to standard non-adaptive auto-regressive inference. This TTC framework marks a foundational step towards more advanced reasoning algorithms or PDE modeling, inluding building reinforcement-learning-based approaches, potentially transforming computational workflows in physics and engineering.

cs.LG↗

arXiv2025

Generalization vs. Memorization in Autoregressive Deep Learning: Or, Examining Temporal Decay of Gradient Coherence

Foundation models trained as autoregressive PDE surrogates hold significant promise for accelerating scientific discovery through their capacity to both extrapolate beyond training regimes and efficiently adapt to downstream tasks despite a paucity of examples for fine-tuning. However, reliably achieving genuine generalization - a necessary capability for producing novel scientific insights and robustly performing during deployment - remains a critical challenge. Establishing whether or not these requirements are met demands evaluation metrics capable of clearly distinguishing genuine model generalization from mere memorization. We apply the influence function formalism to systematically characterize how autoregressive PDE surrogates assimilate and propagate information derived from diverse physical scenarios, revealing fundamental limitations of standard models and training routines in addition to providing actionable insights regarding the design of improved surrogates.

physics.comp-ph↗

arXiv2024

What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning

Differentiable Programming for scientific machine learning (SciML) has recently seen considerable interest and success, as it directly embeds neural networks inside PDEs, often called as NeuralPDEs, derived from first principle physics. Therefore, there is a widespread assumption in the community that NeuralPDEs are more trustworthy and generalizable than black box models. However, like any SciML model, differentiable programming relies predominantly on high-quality PDE simulations as "ground truth" for training. However, mathematics dictates that these are only discrete numerical approximations of the true physics. Therefore, we ask: Are NeuralPDEs and differentiable programming models trained on PDE simulations as physically interpretable as we think? In this work, we rigorously attempt to answer these questions, using established ideas from numerical analysis, experiments, and analysis of model Jacobians. Our study shows that NeuralPDEs learn the artifacts in the simulation training data arising from the discretized Taylor Series truncation error of the spatial derivatives. Additionally, NeuralPDE models are systematically biased, and their generalization capability is likely enabled by a fortuitous interplay of numerical dissipation and truncation error in the training dataset and NeuralPDE, which seldom happens in practical applications. This bias manifests aggressively even in relatively accessible 1-D equations, raising concerns about the veracity of differentiable programming on complex, high-dimensional, real-world PDEs, and in dataset integrity of foundation models. Further, we observe that the initial condition constrains the truncation error in initial-value problems in PDEs, thereby exerting limitations to extrapolation. Finally, we demonstrate that an eigenanalysis of model weights can indicate a priori if the model will be inaccurate for out-of-distribution testing.

cs.LG↗

arXiv2024

Attention-Based Reconstruction of Full-Field Tsunami Waves from Sparse Tsunameter Networks

We investigate the potential of an attention-based neural network architecture, the Senseiver, for sparse sensing in tsunami forecasting. Specifically, we focus on the Tsunami Data Assimilation Method, which generates forecasts from tsunameter networks. Our model is used to reconstruct high-resolution tsunami wavefields from extremely sparse observations, including cases where the tsunami epicenters are not represented in the training set. Furthermore, we demonstrate that our approach significantly outperforms the Linear Interpolation with Huygens-Fresnel Principle in generating dense observation networks, achieving markedly improved accuracy.

cs.LG↗

arXiv2024

Physics-constrained coupled neural differential equations for one dimensional blood flow modeling

Computational cardiovascular flow modeling plays a crucial role in understanding blood flow dynamics. While 3D models provide acute details, they are computationally expensive, especially with fluid-structure interaction (FSI) simulations. 1D models offer a computationally efficient alternative, by simplifying the 3D Navier-Stokes equations through axisymmetric flow assumption and cross-sectional averaging. However, traditional 1D models based on finite element methods (FEM) often lack accuracy compared to 3D averaged solutions. This study introduces a novel physics-constrained machine learning technique that enhances the accuracy of 1D blood flow models while maintaining computational efficiency. Our approach, utilizing a physics-constrained coupled neural differential equation (PCNDE) framework, demonstrates superior performance compared to conventional FEM-based 1D models across a wide range of inlet boundary condition waveforms and stenosis blockage ratios. A key innovation lies in the spatial formulation of the momentum conservation equation, departing from the traditional temporal approach and capitalizing on the inherent temporal periodicity of blood flow. This spatial neural differential equation formulation switches space and time and overcomes issues related to coupling stability and smoothness, while simplifying boundary condition implementation. The model accurately captures flow rate, area, and pressure variations for unseen waveforms and geometries. We evaluate the model's robustness to input noise and explore the loss landscapes associated with the inclusion of different physics terms. This advanced 1D modeling technique offers promising potential for rapid cardiovascular simulations, achieving computational efficiency and accuracy. By combining the strengths of physics-based and data-driven modeling, this approach enables fast and accurate cardiovascular simulations.

physics.flu-dyn↗

arXiv2023

Data-Driven Mori-Zwanzig: Reduced Order Modeling of Sparse Sensors Measurements for Boundary Layer Transition

Understanding, predicting and controlling laminar-turbulent boundary-layer transition is crucial for the next generation aircraft design. However, in real flight experiments, or wind tunnel tests, often only sparse sensor measurements can be collected at fixed locations. Thus, in developing reduced models for predicting and controlling the flow at the sensor locations, the main challenge is in accounting for how the surrounding field of unobserved variables interacts with the observed variables at the fixed sensor locations. This makes the Mori-Zwanzig (MZ) formalism a natural choice, as it results in the Generalized Langevin Equations which provides a framework for constructing non-Markovian reduced-order models that includes the effects the unresolved variables have on the resolved variables. These effects are captured in the so called memory kernel and orthogonal dynamics. In this work, we explore the data-driven MZ formulations to two boundary layer flows obtained from DNS data; a low speed incompressible flow; and a high speed compressible flow over a flared cone at Mach 6. An array of "sensors" are placed near the surface of the solid boundary, and the MZ operators are learned and the predictions are compared to the Extended Dynamic Mode Decomposition (EDMD), both using delay embedded coordinates. Further comparisons are made with Long Short-Term Memory (LSTM) and a regression based projection framework using neural networks for the MZ operators. First we compare the effects of including delay embedded coordinates with EDMD and Mori based MZ and provide evidence that using both memory and delay embedded coordinates minimizes generalization errors on the relevant time scales. Next, we provide numerical evidence that the data-driven regression based projection MZ model performs best with respect to the prediction accuracy (minimum generalization error) on the relevant time scales.

physics.flu-dyn↗

arXiv2023

Data-Driven Mori-Zwanzig: Approaching a Reduced Order Model for Hypersonic Boundary Layer Transition

In this work, we apply, for the first time to spatially inhomogeneous flows, a recently developed data-driven learning algorithm of Mori-Zwanzig (MZ) operators, which is based on a generalized Koopman's description of dynamical systems. The MZ formalism provides a mathematically exact procedure for constructing non-Markovian reduced-order models of resolved variables from high-dimensional dynamical systems, where the effects due to the unresolved dynamics are captured in the memory kernel and orthogonal dynamics. The algorithm developed in this work applies Mori's linear projection operator and an SVD based compression to the selection of the resolved variables (equivalently, a low rank approximation of the two time covariance matrices). We show that this MZ decomposition not only identifies the same spatio-temporal structures found by DMD, but it can also be used to extract spatio-temporal structures of the hysteresis effects present in the memory kernels. We perform an analysis of these structures in the context of a laminar-turbulent boundary-layer transition flow over a flared cone at Mach 6, and show the dynamical relevance of the memory kernels. Additionally, by including these memory terms learned in our data-driven MZ approach, we show improvement in prediction accuracy over DMD at the same level of truncation and at a similar computational cost. Furthermore, an analysis of the spatio-temporal structures of the MZ operators shows identifiable structures associated with the nonlinear generation of the so-called "hot" streaks on the surface of the flared code, which have previously been observed in experiments and direct numerical simulations.

physics.flu-dyn↗

arXiv2023

Machine Learning technique for isotopic determination of radioisotopes using HPGe $\mathrm{\gamma}$-ray spectra

$\mathrm{\gamma}$-ray spectroscopy is a quantitative, non-destructive technique that may be utilized for the identification and quantitative isotopic estimation of radionuclides. Traditional methods of isotopic determination have various challenges that contribute to statistical and systematic uncertainties in the estimated isotopics. Furthermore, these methods typically require numerous pre-processing steps, and have only been rigorously tested in laboratory settings with limited shielding. In this work, we examine the application of a number of machine learning based regression algorithms as alternatives to conventional approaches for analyzing $\mathrm{\gamma}$-ray spectroscopy data in the Emergency Response arena. This approach not only eliminates many steps in the analysis procedure, and therefore offers potential to reduce this source of systematic uncertainty, but is also shown to offer comparable performance to conventional approaches in the Emergency Response Application.

physics.data-an↗

arXiv2019

Time-series learning of latent-space dynamics for reduced-order model closure

We study the performance of long short-term memory networks (LSTMs) and neural ordinary differential equations (NODEs) in learning latent-space representations of dynamical equations for an advection-dominated problem given by the viscous Burgers equation. Our formulation is devised in a non-intrusive manner with an equation-free evolution of dynamics in a reduced space with the latter being obtained through a proper orthogonal decomposition. In addition, we leverage the sequential nature of learning for both LSTMs and NODEs to demonstrate their capability for closure in systems which are not completely resolved in the reduced space. We assess our hypothesis for two advection-dominated problems given by the viscous Burgers equation. It is observed that both LSTMs and NODEs are able to reproduce the effects of the absent scales for our test cases more effectively than intrusive dynamics evolution through a Galerkin projection. This result empirically suggests that time-series learning techniques implicitly leverage a memory kernel for coarse-grained system closure as is suggested through the Mori-Zwanzig formalism.

physics.comp-ph↗

arXiv2019

Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence

High-fidelity modeling of turbulent flows is one of the major challenges in computational physics, with diverse applications in engineering, earth sciences and astrophysics, among many others. The rising popularity of high-fidelity computational fluid dynamics (CFD) techniques like direct numerical simulation (DNS) and large eddy simulation (LES) have made significant inroads into the problem. However, they remain out of reach for many practical three-dimensional flows characterized by extremely large domains and transient phenomena. Therefore designing efficient and accurate data-driven generative approaches to model turbulence is a necessity. We propose a novel training approach for dimensionality reduction and spatio-temporal modeling of the three-dimensional dynamics of turbulence using a combination of Convolutional autoencoder and the Convolutional LSTM neural networks. The quality of the emulated turbulent fields is assessed with rigorous physics-based statistical tests, instead of visual assessments. The results show significant promise in the training methodology to generate physically consistent turbulent flows at a small fraction of the computing resources required for DNS.

physics.flu-dyn↗

arXiv2019

The ISTI Rapid Response on Exploring Cloud Computing 2018

This report describes eighteen projects that explored how commercial cloud computing services can be utilized for scientific computation at national laboratories. These demonstrations ranged from deploying proprietary software in a cloud environment to leveraging established cloud-based analytics workflows for processing scientific datasets. By and large, the projects were successful and collectively they suggest that cloud computing can be a valuable computational resource for scientific computation at national laboratories.

cs.DC↗

arXiv2018

From Deep to Physics-Informed Learning of Turbulence: Diagnostics

We describe tests validating progress made toward acceleration and automation of hydrodynamic codes in the regime of developed turbulence by three Deep Learning (DL) Neural Network (NN) schemes trained on Direct Numerical Simulations of turbulence. Even the bare DL solutions, which do not take into account any physics of turbulence explicitly, are impressively good overall when it comes to qualitative description of important features of turbulence. However, the early tests have also uncovered some caveats of the DL approaches. We observe that the static DL scheme, implementing Convolutional GAN and trained on spatial snapshots of turbulence, fails to reproduce intermittency of turbulent fluctuations at small scales and details of the turbulence geometry at large scales. We show that the dynamic NN schemes, namely LAT-NET and Compressed Convolutional LSTM, trained on a temporal sequence of turbulence snapshots are capable to correct for the caveats of the static NN. We suggest a path forward towards improving reproducibility of the large-scale geometry of turbulence with NN.

physics.flu-dyn↗