arXiv subjects

George Em Karniadakis

Publications and source records attributed to George Em Karniadakis.

At least 217 records · Page 12Linked to original sources

arXiv2020

Non-invasive Inference of Thrombus Material Properties with Physics-informed Neural Networks

We employ physics-informed neural networks (PINNs) to infer properties of biological materials using synthetic data. In particular, we successfully apply PINNs on inferring the thrombus permeability and visco-elastic modulus from thrombus deformation data, which can be described by the fourth-order Cahn-Hilliard and Navier-Stokes Equations. In PINNs, the partial differential equations are encoded into the loss function, where partial derivatives can be obtained through automatic differentiation (AD). In addition, to tackling the challenge of calculating the fourth-order derivative in the Cahn-Hilliard equation with AD, we introduce an auxiliary network along with the main neural network to approximate the second-derivative of the energy potential term. Our model can predict simultaneously unknown parameters and velocity, pressure, and deformation gradient fields by merely training with partial information among all data, i.e., phase-field and pressure measurements, and is also highly flexible in sampling within the spatio-temporal domain for data acquisition. We validate our model by numerical solutions from the spectral/\textit{hp} element method (SEM) and demonstrate its robustness by training it with noisy measurements. Our results show that PINNs can accurately infer the material properties with noisy synthetic data, and thus they have great potential for inferring these properties from experimental multi-modality and multi-fidelity data.

physics.comp-ph↗

arXiv2020

Active- and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows

Active- and transfer-learning are applied to polymer flows for the multiscale discovery of effective constitutive approximations required in viscoelastic flow simulation. The result is macroscopic rheology directly connected to a microstructural model. Micro and macroscale simulations are adaptively coupled by means of Gaussian process regression to run the expensive microscale computations only as necessary. This active-learning guided multiscale method can automatically detect the inaccuracy of the learned constitutive closure and initiate simulations at new sampling points informed by proper acquisition functions, leading to an autonomic microscale-macroscale coupled system. Also, we develop a new dissipative particle dynamics model with the range of interaction cutoff between particles allowed to vary with the local strain-rate invariant, which is able to capture both the shear-thinning viscosity and the normal stress difference functions consistent with rheological experiments for aqueous polyacrylamide solutions. Our numerical experiments demonstrate the effectiveness of using active- and transfer-learning schemes to on-the-fly couple a spectral element solver and a mesoscopic particle-based simulator, and verify that the microscale-macroscale coupled model with effective constitutive closure learned from microscopic dynamics can outperform empirical constitutive models compared to experimental observations. The effective closure learned in a channel simulation is then transferred directly to the flow past a circular cylinder, where the results show that only two additional microscopic simulations are required to achieve a satisfactory constitutive model to once again close the continuum equations. This new paradigm of active- and transfer-learning for multiscale modeling is readily applicable to other microscale-macroscale coupled simulations of complex fluids and other materials.

physics.comp-ph↗

arXiv2020

Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks

We introduce an optimized physics-informed neural network (PINN) trained to solve the problem of identifying and characterizing a surface breaking crack in a metal plate. PINNs are neural networks that can combine data and physics in the learning process by adding the residuals of a system of Partial Differential Equations to the loss function. Our PINN is supervised with realistic ultrasonic surface acoustic wave data acquired at a frequency of 5 MHz. The ultrasonic surface wave data is represented as a surface deformation on the top surface of a metal plate, measured by using the method of laser vibrometry. The PINN is physically informed by the acoustic wave equation and its convergence is sped up using adaptive activation functions. The adaptive activation function uses a scalable hyperparameter in the activation function, which is optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The usage of adaptive activation function significantly improves the convergence, notably observed in the current study. We use PINNs to estimate the speed of sound of the metal plate, which we do with an error of 1\%, and then, by allowing the speed of sound to be space dependent, we identify and characterize the crack as the positions where the speed of sound has decreased. Our study also shows the effect of sub-sampling of the data on the sensitivity of sound speed estimates. More broadly, the resulting model shows a promising deep neural network model for ill-posed inverse problems.

cs.LG↗

arXiv2020

DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators

While it is widely known that neural networks are universal approximators of continuous functions, a less known and perhaps more powerful result is that a neural network with a single hidden layer can approximate accurately any nonlinear continuous operator. This universal approximation theorem is suggestive of the potential application of neural networks in learning nonlinear operators from data. However, the theorem guarantees only a small approximation error for a sufficient large network, and does not consider the important optimization and generalization errors. To realize this theorem in practice, we propose deep operator networks (DeepONets) to learn operators accurately and efficiently from a relatively small dataset. A DeepONet consists of two sub-networks, one for encoding the input function at a fixed number of sensors $x_i, i=1,\dots,m$ (branch net), and another for encoding the locations for the output functions (trunk net). We perform systematic simulations for identifying two types of operators, i.e., dynamic systems and partial differential equations, and demonstrate that DeepONet significantly reduces the generalization error compared to the fully-connected networks. We also derive theoretically the dependence of the approximation error in terms of the number of sensors (where the input function is defined) as well as the input function type, and we verify the theorem with computational results. More importantly, we observe high-order error convergence in our computational tests, namely polynomial rates (from half order to fourth order) and even exponential convergence with respect to the training dataset size.

cs.LG↗

arXiv2020

Trainability of ReLU networks and Data-dependent Initialization

In this paper, we study the trainability of rectified linear unit (ReLU) networks. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of neurons are introduced; tentative and permanent death. A network is then said to be trainable if the number of permanently dead neurons is sufficiently small for a learning task. We refer to the probability of a network being trainable as trainability. We show that a network being trainable is a necessary condition for successful training and the trainability serves as an upper bound of successful training rates. In order to quantify the trainability, we study the probability distribution of the number of active neurons at the initialization. In many applications, over-specified or over-parameterized neural networks are successfully employed and shown to be trained effectively. With the notion of trainability, we show that over-parameterization is both a necessary and a sufficient condition for minimizing the training loss. Furthermore, we propose a data-dependent initialization method in the over-parameterized setting. Numerical examples are provided to demonstrate the effectiveness of the method and our theoretical findings.

cs.LG↗

arXiv2020

Physics-informed neural networks for inverse problems in nano-optics and metamaterials

In this paper we employ the emerging paradigm of physics-informed neural networks (PINNs) for the solution of representative inverse scattering problems in photonic metamaterials and nano-optics technologies. In particular, we successfully apply mesh-free PINNs to the difficult task of retrieving the effective permittivity parameters of a number of finite-size scattering systems that involve many interacting nanostructures as well as multi-component nanoparticles. Our methodology is fully validated by numerical simulations based on the Finite Element Method (FEM). The development of physics-informed deep learning techniques for inverse scattering can enable the design of novel functional nanostructures and significantly broaden the design space of metamaterials by naturally accounting for radiation and finite-size effects beyond the limitations of traditional effective medium theories.

physics.comp-ph↗

arXiv2020

B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINN posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

stat.ML↗

arXiv2020

NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations

We employ physics-informed neural networks (PINNs) to simulate the incompressible flows ranging from laminar to turbulent flows. We perform PINN simulations by considering two different formulations of the Navier-Stokes equations: the velocity-pressure (VP) formulation and the vorticity-velocity (VV) formulation. We refer to these specific PINNs for the Navier-Stokes flow nets as NSFnets. Analytical solutions and direct numerical simulation (DNS) databases provide proper initial and boundary conditions for the NSFnet simulations. The spatial and temporal coordinates are the inputs of the NSFnets, while the instantaneous velocity and pressure fields are the outputs for the VP-NSFnet, and the instantaneous velocity and vorticity fields are the outputs for the VV-NSFnet. These two different forms of the Navier-Stokes equations together with the initial and boundary conditions are embedded into the loss function of the PINNs. No data is provided for the pressure to the VP-NSFnet, which is a hidden state and is obtained via the incompressibility constraint without splitting the equations. We obtain good accuracy of the NSFnet simulation results upon convergence of the loss function, verifying that NSFnets can effectively simulate complex incompressible flows using either the VP or the VV formulations. We also perform a systematic study on the weights used in the loss function for the data/physics components and investigate a new way of computing the weights dynamically to accelerate training and enhance accuracy. Our results suggest that the accuracy of NSFnets, for both laminar and turbulent flows, can be improved with proper tuning of weights (manual or dynamic) in the loss function.

physics.comp-ph↗

arXiv2020

hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition

We formulate a general framework for hp-variational physics-informed neural networks (hp-VPINNs) based on the nonlinear approximation of shallow and deep neural networks and hp-refinement via domain decomposition and projection onto space of high-order polynomials. The trial space is the space of neural network, which is defined globally over the whole computational domain, while the test space contains the piecewise polynomials. Specifically in this study, the hp-refinement corresponds to a global approximation with local learning algorithm that can efficiently localize the network parameter optimization. We demonstrate the advantages of hp-VPINNs in accuracy and training cost for several numerical examples of function approximation and solving differential equations.

cs.NE↗

arXiv2020

Reinforcement Learning for Active Flow Control in Experiments

We demonstrate experimentally the feasibility of applying reinforcement learning (RL) in flow control problems by automatically discovering active control strategies without any prior knowledge of the flow physics. We consider the turbulent flow past a circular cylinder with the aim of reducing the cylinder drag force or maximizing the power gain efficiency by properly selecting the rotational speed of two small diameter cylinders, parallel to and located downstream of the larger cylinder. Given properly designed rewards and noise reduction techniques, after tens of towing experiments, the RL agent could discover the optimal control strategy, comparable to the optimal static control. While RL has been found to be effective in recent computer flow simulation studies, this is the first time that its effectiveness is demonstrated experimentally, paving the way for exploring new optimal active flow control strategies in complex fluid mechanics applications.

physics.flu-dyn↗

arXiv2020

Controlled release of entrapped nanoparticles from thermoresponsive hydrogels with tunable network characteristics

Thermoresponsive hydrogels have been studied intensively for creating smart drug carriers and controlled drug delivery. Understanding the drug release kinetics and corresponding transport mechanisms of nanoparticles (NPs) in a thermoresponsive hydrogel network is the key to the successful design of a smart drug delivery system. We construct a mesoscopic model of rigid NPs entrapped in a hydrogel network in an aqueous solution, where the hydrogel network is formed by cross-linked semiflexible polymers of PNIPAM. By varying the environmental temperature crossing the lower critical solution temperature of PNIPAM we can significantly change the hydrogel network characteristics. We systematically investigate how the matrix porosity and the nanoparticle size affect the NPs' transport kinetics at different temperatures. Quantitative results on the mean-squared displacement and the van Hove displacement distributions of NPs show that all NPs entrapped in the smart hydrogels undergo subdiffusion at both low and high temperatures. For a coil state, the subdiffusive exponent and the diffusion coefficient of NPs increase due to the increased kinetic energy and the decreased confinement on NPs, while the transport of NPs in the hydrogels can be also enhanced by decreasing the matrix porosity and NPs' size. However, when the solution temperature is increased above the critical temperature, the hydrogel network collapses following the coil-to-globule transition, with the NPs tightly trapped in some local regions inside the hydrogels. Consequently, the NP diffusion coefficient can be reduced by two orders of magnitude, or the diffusion processes can even be completely stopped. These findings provide new insights for designing controlled drug release from stimuli-responsive hydrogels, including autonomously switch on/off drug release to respond to the changes of the local environment.

cond-mat.soft↗

arXiv2019

Physics-informed semantic inpainting: Application to geostatistical modeling

A fundamental problem in geostatistical modeling is to infer the heterogeneous geological field based on limited measurements and some prior spatial statistics. Semantic inpainting, a technique for image processing using deep generative models, has been recently applied for this purpose, demonstrating its effectiveness in dealing with complex spatial patterns. However, the original semantic inpainting framework incorporates only information from direct measurements, while in geostatistics indirect measurements are often plentiful. To overcome this limitation, here we propose a physics-informed semantic inpainting framework, employing the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) and jointly incorporating the direct and indirect measurements by exploiting the underlying physical laws. Our simulation results for a high-dimensional problem with 512 dimensions show that in the new method, the physical conservation laws are satisfied and contribute in enhancing the inpainting performance compared to using only the direct measurements.

eess.IV↗

arXiv2019

What Is the Fractional Laplacian?

The fractional Laplacian in R^d has multiple equivalent characterizations. Moreover, in bounded domains, boundary conditions must be incorporated in these characterizations in mathematically distinct ways, and there is currently no consensus in the literature as to which definition of the fractional Laplacian in bounded domains is most appropriate for a given application. The Riesz (or integral) definition, for example, admits a nonlocal boundary condition, where the value of a function u(x) must be prescribed on the entire exterior of the domain in order to compute its fractional Laplacian. In contrast, the spectral definition requires only the standard local boundary condition. These differences, among others, lead us to ask the question: "What is the fractional Laplacian?" We compare several commonly used definitions of the fractional Laplacian (the Riesz, spectral, directional, and horizon-based nonlocal definitions), and we use a joint theoretical and computational approach to examining their different characteristics by studying solutions of related fractional Poisson equations formulated on bounded domains. In this work, we provide new numerical methods as well as a self-contained discussion of state-of-the-art methods for discretizing the fractional Laplacian, and we present new results on the differences in features, regularity, and boundary behaviors of solutions to equations posed with these different definitions. We present stochastic interpretations and demonstrate the equivalence between some recent formulations. Through our efforts, we aim to further engage the research community in open problems and assist practitioners in identifying the most appropriate definition and computational approach to use for their mathematical models in addressing anomalous transport in diverse applications.

math.NA↗

arXiv2019

Learning and Meta-Learning of Stochastic Advection-Diffusion-Reaction Systems from Sparse Measurements

Physics-informed neural networks (PINNs) were recently proposed in [1] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a nonlinear advection-diffusion-reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimize the hyper-parameters of sPINN by using the Bayesian optimization method (meta-learning), and compare the results with the empirically selected hyper-parameters of sPINN.

physics.comp-ph↗

arXiv2019

PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs

Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time-space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speedup for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion-reaction equation. Our results demonstrate that PPINNs converge in a couple of iterations with significant speed-ups proportional to the number of time-subdomains employed.

physics.comp-ph↗

arXiv2019

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks

One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs). We address this problem by taking advantage of recent advances in scientific machine learning and the dynamically orthogonal (DO) and bi-orthogonal (BO) methods for representing stochastic processes. Specifically, we propose two new Physics-Informed Neural Networks (PINNs) for solving time-dependent SPDEs, namely the NN-DO/BO methods, which incorporate the DO/BO constraints into the loss function with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the proposed methods overcome some of the drawbacks of the original DO/BO methods: we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and computational complexity as for forward problems. We demonstrate the capability of the proposed methods via several numerical examples: (1) A linear stochastic advection equation with deterministic initial condition where the original DO/BO method would fail; (2) Long-time integration of the stochastic Burgers' equation with many eigenvalue crossings during the whole time evolution where the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward and the inverse problem, including noisy initial data, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems.

cs.LG↗

arXiv2019

Potential Flow Generator with $L_2$ Optimal Transport Regularity for Generative Models

We propose a potential flow generator with $L_2$ optimal transport regularity, which can be easily integrated into a wide range of generative models including different versions of GANs and flow-based models. We show the correctness and robustness of the potential flow generator in several 2D problems, and illustrate the concept of "proximity" due to the $L_2$ optimal transport regularity. Subsequently, we demonstrate the effectiveness of the potential flow generator in image translation tasks with unpaired training data from the MNIST dataset and the CelebA dataset.

cs.LG↗

arXiv2019

Density-Dependent Finite System-Size Effects in Equilibrium Molecular Dynamics Estimation of Shear Viscosity: Hydrodynamic and Configurational Study

We study the intrinsic nature of the finite system-size effect in estimating shear viscosity of dilute and dense fluids within the framework of the Green-Kubo approach. From extensive molecular dynamics simulations, we observe that the size effect on shear viscosity is characterized by an oscillatory behavior with respect to system size $L$ at high density and by a scaling behavior with an $L^{-1}$ correction term at low density. Analysis of the potential contribution in the shear-stress autocorrelation function reveals that the former is configurational and is attributed to the inaccurate description of the long-range spatial correlations in finite systems. Observation of the long-time inverse-power decay in the kinetic contribution confirms its hydrodynamic nature. The $L^{-1}$ correction term of shear viscosity is explained by the sensitive change in the long-time tail obtained from a finite system.

physics.chem-ph↗