arXiv subjects

Panos Stinis

Publications and source records attributed to Panos Stinis.

70 records · Page 4Linked to original sources

arXiv2019

A Kinetic Monte Carlo Approach for Simulating Cascading Transmission Line Failure

In this work, cascading transmission line failures are studied through a dynamical model of the power system operating under fixed conditions. The power grid is modeled as a stochastic dynamical system where first-principles electromechanical dynamics are excited by small Gaussian disturbances in demand and generation around a specified operating point. In this context, a single line failure is interpreted in a large deviation context as a first escape event across a surface in phase space defined by line security constraints. The resulting system of stochastic differential equations admits a transverse decomposition of the drift, which leads to considerable simplification in evaluating the quasipotential (rate function) and, consequently, computation of exit rates. Tractable expressions for the rate of transmission line failure in a restricted network are derived from large deviation theory arguments and validated against numerical simulations. Extensions to realistic settings are considered, and individual line failure models are aggregated into a Markov model of cascading failure inspired by chemical kinetics. Cascades are generated by traversing a graph composed of weighted edges representing transitions to degraded network topologies. Numerical results indicate that the Markov model can produce cascades with qualitative power-law properties similar to those observed in empirical cascades.

eess.SY↗

arXiv2019

Improving solution accuracy and convergence for stochastic physics parameterizations with colored noise

Stochastic parameterizations are used in numerical weather prediction and climate modeling to help capture the uncertainty in the simulations and improve their statistical properties. Convergence issues can arise when time integration methods originally developed for deterministic differential equations are applied naively to stochastic problems. (Hodyss et al 2013, 2014) demonstrated that a correction term to various deterministic numerical schemes, known in stochastic analysis as the Itô correction, can help improve solution accuracy and ensure convergence to the physically relevant solution without substantial computational overhead. The usual formulation of the Itô correction is valid only when the stochasticity is represented by {\it white} noise. In this study, a generalized formulation of the Itô correction is derived for noises of any color. The formulation is applied to a test problem described by an advection-diffusion equation forced with a spectrum of fast processes. We present numerical results for cases with both constant and spatially varying advection velocities to show that, for the same time step sizes, the introduction of the generalized Itô correction helps to substantially reduce time integration error and significantly improve the convergence rate of the numerical solutions when the forcing term in the governing equation is rough (fast varying); alternatively, for the same target accuracy, the generalized Itô correction allows for the use of significantly longer time steps and hence helps to reduce the computational cost of the numerical simulation.

math.NA↗

arXiv2019

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

We suggest ways to enforce given constraints in the output of a Generative Adversarial Network (GAN) generator both for interpolation and extrapolation (prediction). For the case of dynamical systems, given a time series, we wish to train GAN generators that can be used to predict trajectories starting from a given initial condition. In this setting, the constraints can be in algebraic and/or differential form. Even though we are predominantly interested in the case of extrapolation, we will see that the tasks of interpolation and extrapolation are related. However, they need to be treated differently. For the case of interpolation, the incorporation of constraints is built into the training of the GAN. The incorporation of the constraints respects the primary game-theoretic setup of a GAN so it can be combined with existing algorithms. However, it can exacerbate the problem of instability during training that is well-known for GANs. We suggest adding small noise to the constraints as a simple remedy that has performed well in our numerical experiments. The case of extrapolation (prediction) is more involved. During training, the GAN generator learns to interpolate a noisy version of the data and we enforce the constraints. This approach has connections with model reduction that we can utilize to improve the efficiency and accuracy of the training. Depending on the form of the constraints, we may enforce them also during prediction through a projection step. We provide examples of linear and nonlinear systems of differential equations to illustrate the various constructions.

cs.LG↗

arXiv2019

Enforcing constraints for time series prediction in supervised, unsupervised and reinforcement learning

We assume that we are given a time series of data from a dynamical system and our task is to learn the flow map of the dynamical system. We present a collection of results on how to enforce constraints coming from the dynamical system in order to accelerate the training of deep neural networks to represent the flow map of the system as well as increase their predictive ability. In particular, we provide ways to enforce constraints during training for all three major modes of learning, namely supervised, unsupervised and reinforcement learning. In general, the dynamic constraints need to include terms which are analogous to memory terms in model reduction formalisms. Such memory terms act as a restoring force which corrects the errors committed by the learned flow map during prediction. For supervised learning, the constraints are added to the objective function. For the case of unsupervised learning, in particular generative adversarial networks, the constraints are introduced by augmenting the input of the discriminator. Finally, for the case of reinforcement learning and in particular actor-critic methods, the constraints are added to the reward function. In addition, for the reinforcement learning case, we present a novel approach based on homotopy of the action-value function in order to stabilize and accelerate training. We use numerical results for the Lorenz system to illustrate the various constructions.

cs.LG↗

arXiv2019

A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations

We investigate the use of discrete and continuous versions of physics-informed neural network methods for learning unknown dynamics or constitutive relations of a dynamical system. For the case of unknown dynamics, we represent all the dynamics with a deep neural network (DNN). When the dynamics of the system are known up to the specification of constitutive relations (that can depend on the state of the system), we represent these constitutive relations with a DNN. The discrete versions combine classical multistep discretization methods for dynamical systems with neural network based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of the whole dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information is available into an algorithm can enhance its efficiency and/or accuracy.

cs.LG↗

arXiv2019

A data-driven framework for sparsity-enhanced surrogates with arbitrary mutually dependent randomness

The challenge of quantifying uncertainty propagation in real-world systems is rooted in the high-dimensionality of the stochastic input and the frequent lack of explicit knowledge of its probability distribution. Traditional approaches show limitations for such problems. To address these difficulties, we have developed a general framework of constructing surrogate models on spaces of stochastic input with arbitrary probability measure irrespective of the mutual dependencies between individual components and the analytical form. The present Data-driven Sparsity-enhancing Rotation for Arbitrary Randomness (DSRAR) framework includes a data-driven construction of multivariate polynomial basis for arbitrary mutually dependent probability measure and a sparsity enhancement rotation procedure. This sparsity-enhancing rotation method was initially proposed in our previous work [1] for Gaussian distributions, which may not be feasible for non-Gaussian distributions due to the loss of orthogonality after the rotation. To remedy such difficulties, we developed the new approach to construct orthonormal polynomials for arbitrary mutually dependent (amdP) randomness, ensuring the constructed basis maintains the orthogonality with respect to the density of the rotated random vector, where directly applying the regular polynomial chaos including arbitrary polynomial chaos (aPC) [2] shows limitations due to the assumption of the mutual independence between the components of the random inputs. The developed DSRAR framework leads to accurate recovery of a sparse representation of the target functions. The effectiveness of our method is demonstrated in challenging problems such as PDEs and realistic molecular systems where the underlying density is implicitly represented by a large collection of sample data, as well as systems with explicitly given non-Gaussian probabilistic measures.

math.NA↗

arXiv2018

Renormalization and blow-up for the 3D Euler equations

In recent work we have developed a renormalization framework for stabilizing reduced order models for time-dependent partial differential equations. We have applied this framework to the open problem of finite-time singularity formation (blow-up) for the 3D Euler equations of incompressible fluid flow. The renormalized coefficients in the reduced order models decay algebraically with time and resolution. Our results for the behavior of the solutions are consistent with the formation of a finite-time singularity.

math.NA↗

arXiv2018

Doing the impossible: Why neural networks can be trained at all

As deep neural networks grow in size, from thousands to millions to billions of weights, the performance of those networks becomes limited by our ability to accurately train them. A common naive question arises: if we have a system with billions of degrees of freedom, don't we also need billions of samples to train it? Of course, the success of deep learning indicates that reliable models can be learned with reasonable amounts of data. Similar questions arise in protein folding, spin glasses and biological neural networks. With effectively infinite potential folding/spin/wiring configurations, how does the system find the precise arrangement that leads to useful and robust results? Simple sampling of the possible configurations until an optimal one is reached is not a viable option even if one waited for the age of the universe. On the contrary, there appears to be a mechanism in the above phenomena that forces them to achieve configurations that live on a low-dimensional manifold, avoiding the curse of dimensionality. In the current work we use the concept of mutual information between successive layers of a deep neural network to elucidate this mechanism and suggest possible ways of exploiting it to accelerate training. We show that adding structure to the neural network that enforces higher mutual information between layers speeds training and leads to more accurate results. High mutual information between layers implies that the effective number of free parameters is exponentially smaller than the raw number of tunable weights.

cs.LG↗

arXiv2018

Mori-Zwanzig reduced models for uncertainty quantification

In many time-dependent problems of practical interest the parameters and/or initial conditions entering the equations describing the evolution of the various quantities exhibit uncertainty. One way to address the problem of how this uncertainty impacts the solution is to expand the solution using polynomial chaos expansions and obtain a system of differential equations for the evolution of the expansion coefficients. We present an application of the Mori-Zwanzig (MZ) formalism to the problem of constructing reduced models of such systems of differential equations. In particular, we construct reduced models for a subset of the polynomial chaos expansion coefficients that are needed for a full description of the uncertainty caused by uncertain parameters or initial conditions. Even though the MZ formalism is exact, its straightforward application to the problem of constructing reduced models for estimating uncertainty involves the computation of memory terms whose cost can become prohibitively expensive. For those cases, we present a Markovian reformulation of the MZ formalism which can lead to approximations that can alleviate some of the computational expense while retaining an accuracy advantage over reduced models that discard the memory altogether. Our results support the conclusion that successful reduced models need to include memory effects.

math.NA↗

arXiv2017

Dynamic Looping of a Free-Draining Polymer

We revisit the celebrated Wilemski-Fixman (WF) treatment for the looping time of a free-draining polymer. The WF theory introduces a sink term into the Fokker-Planck equation for the $3(N+1)$-dimensional Ornstein-Uhlenbeck process of the polymer dynamics, which accounts for the appropriate boundary condition due to the formation of a loop. The assumption for WF theory is considerably relaxed. A perturbation method approach is developed that justifies and generalizes the previous results using either a Delta sink or a Heaviside sink. For both types of sinks, we show that under the condition of a small dimensionless $ε$, the ratio of capture radius to the Kuhn length, we are able to systematically produce all known analytical and asymptotic results obtained by other methods. This includes most notably the transition regime between the $N^2$ scaling of Doi, and $N\sqrt{N}/ε$ scaling of Szabo, Schulten, and Schulten. The mathematical issue at play is the non-uniform convergence of $ε\to 0$ and $N\to\infty$, the latter being an inherent part of the theory of a Gaussian polymer. Our analysis yields a novel term in the analytical expression for the looping time with small $ε$, which is previously unknown. Monte Carlo numerical simulations corroborate the analytical findings. The systematic method developed here can be applied to other systems modeled by multi-dimensional Smoluchowski equations.

cond-mat.soft↗

arXiv2017

Solving differential equations with unknown constitutive relations as recurrent neural networks

We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and we use recurrent neural network to "learn" the reaction rate from this data. This is achieved by including a discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow's recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differences in design and function require modifications to the conventional wisdom about training such networks.

cs.LG↗

arXiv2017

Stochastic basis adaptation and spatial domain decomposition for PDEs with random coefficients

We present a novel uncertainty quantification approach for high-dimensional stochastic partial differential equations that reduces the computational cost of polynomial chaos methods by decomposing the computational domain into non-overlapping subdomains and adapting the stochastic basis in each subdomain so the local solution has a lower dimensional random space representation. The local solutions are coupled using the Neumann-Neumann algorithm, where we first estimate the interface solution then evaluate the interior solution in each subdomain using the interface solution as a boundary condition. The interior solutions in each subdomain are computed independently of each other, which reduces the operation count from $O(N^α)$ to $O(M^α),$ where $N$ is the total number of degrees of freedom, $M$ is the number of degrees of freedom in each subdomain, and the exponent $α>1$ depends on the uncertainty quantification method used. In addition, the localized nature of solutions makes the proposed approach highly parallelizable. We illustrate the accuracy and efficiency of the approach for linear and nonlinear differential equations with random coefficients.

math.NA↗

arXiv2017

Renormalized Reduced Order Models with Memory for Long Time Prediction

We examine the challenging problem of constructing reduced models for the long time prediction of systems where there is no timescale separation between the resolved and unresolved variables. In previous work we focused on the case where there was only transfer of activity (e.g. energy, mass) from the resolved to the unresolved variables. Here we investigate the much more difficult case where there is two-way transfer of activity between the resolved and unresolved variables. Like in the case of activity drain out of the resolved variables, even if one starts with an exact formalism, like the Mori-Zwanzig (MZ) formalism, the constructed reduced models can become unstable. We show how to remedy this situation by using dynamic information from the full system to renormalize the MZ reduced models. In addition to being stabilized, the renormalized models can be accurate for very long times. We use the Korteweg-de Vries equation to illustrate the approach. The coefficients of the renormalized models exhibit rich structure, including algebraic time dependence and incomplete similarity.

math.NA↗

arXiv2016

Basis adaptation and domain decomposition for steady partial differential equations with random coefficients

We present a novel approach for solving steady-state stochastic partial differential equations (PDEs) with high-dimensional random parameter space. The proposed approach combines spatial domain decomposition with basis adaptation for each subdomain. The basis adaptation is used to address the curse of dimensionality by constructing an accurate low-dimensional representation of the stochastic PDE solution (probability density function and/or its leading statistical moments) in each subdomain. Restricting the basis adaptation to a specific subdomain affords finding a locally accurate solution. Then, the solutions from all of the subdomains are stitched together to provide a global solution. We support our construction with numerical experiments for a steady-state diffusion equation with a random spatially dependent coefficient. Our results show that highly accurate global solutions can be obtained with significantly reduced computational costs.

math.NA↗

arXiv2015

Efficient failure probability calculation through mesh refinement

We present a novel way of accelerating hybrid surrogate methods for the calculation of failure probabilities. The main idea is to use mesh refinement in order to obtain improved local surrogates of low computation cost to simulate on. These improved surrogates can reduce significantly the required number of evaluations of the exact model (which is the usual bottleneck of failure probability calculations). Meanwhile the effort on evaluations of surrogates is dramatically reduced by utilizing low order local surrogates. Numerical results of the application of the proposed approach in several examples of increasing complexity show the robustness, versatility and gain in efficiency of the method.

math.NA↗

arXiv2015

A unified framework for mesh refinement in random and physical space

In recent work we have shown how an accurate reduced model can be utilized to perform mesh refinement in random space. That work relied on the explicit knowledge of an accurate reduced model which is used to monitor the transfer of activity from the large to the small scales of the solution. Since this is not always available, we present in the current work a framework which shares the merits and basic idea of the previous approach but does not require an explicit knowledge of a reduced model. Moreover, the current framework can be applied for refinement in both random and physical space. In this manuscript we focus on the application to random space mesh refinement. We study examples of increasing difficulty (from ordinary to partial differential equations) which demonstrate the efficiency and versatility of our approach. We also provide some results from the application of the new framework to physical space mesh refinement.

math.NA↗