arXiv subjects

Diyora Salimova

Publications and source records attributed to Diyora Salimova.

14 recordsLinked to original sources

arXiv2026

Residual neural networks overcome the curse of dimensionality for semilinear heat equations

Rigorous results show that feedforward neural networks can overcome the curse of dimensionality in the numerical approximation of high-dimensional partial differential equations (PDEs), but comparatively little is known about residual neural networks (ResNets) in the nonlinear PDE setting. We prove that ResNets overcome the curse of dimensionality in the numerical approximation of solutions of semilinear heat equations with globally Lipschitz continuous, gradient-independent nonlinearities: under polynomial growth and network approximability hypotheses on the PDE data, there exist $η\in(0,\infty)$ and ResNets $Ψ_{d,\varepsilon}$, $d\in\mathbb{N}$, $\varepsilon\in(0,1]$, with at most $ηd^η\varepsilon^{-η}$ parameters whose realizations approximate the solution in dimension $d$ with an $L^2$-error of at most $\varepsilon$. The proof represents one deterministic realization of a multilevel Picard estimator by a ResNet whose shortcut connections transmit the spatial variable and a scalar accumulator, while the residual branches successively add the summands of the estimator. For ridge-sum initial conditions, admissible sigmoidal activations, and globally Lipschitz truncations of the nonlinearity, we obtain, for every $ξ>0$, the explicit bound $C_ξd^{4+ξ}\varepsilon^{-(3+ξ)}$ on the number of parameters.

math.NA↗

arXiv2026

Landscape analysis for shallow neural networks: Complete classification of critical points for cubic activation and affine target functions

In this paper, we study the optimization landscape induced by the true loss for shallow polynomial neural networks (PNNs) with $\mathfrak{h} \in \mathbb{N}$ neurons on the hidden layer, one-dimensional input and output layers, and a monomial activation of degree $d \in \mathbb{N}$, trained against a non-constant affine linear target function. Our first main result provides for arbitrary activation degree $d$ a sharp existence/non-existence criterion for \emph{global minimizers} with necessary structural conditions. We show that the infimum of the loss is always zero and achievable with at least $d$ active and visible hidden neurons -- that is, hidden neurons with non-zero inner and outer weights -- with pairwise distinct pivots. In contrast, if $\mathfrak{h} < d$, then the infimum cannot be attained and any minimizing sequence of parameters necessarily diverges to infinity. In the second main result, we provide a complete classification of all critical points of the loss function for the cubic activation. We show that the loss landscape admits no \emph{local maximizers}, critical points cannot have exactly two distinct pivots, global minimizers require at least three distinct pivots, critical points with no active hidden neurons correspond to \emph{saddle points} only, and consequently, \emph{non-global local minimizers} and non-trivial saddle points arise only in networks where all pivots coincide. Moreover, non-global local minimizers require all hidden neurons to be active and visible with exactly one hidden neuron having a slope sign matching that of the target function. Our second main result also guarantees that each hidden neuron of a critical point that is not a global minimizer has either input-dependent or zero contribution, but has no nonzero input-independent contribution, to its corresponding realization function.

math.OC↗

arXiv2025

Refining Diffusion Models for Motion Synthesis with an Acceleration Loss to Generate Realistic IMU Data

We propose a text-to-IMU (inertial measurement unit) motion-synthesis framework to obtain realistic IMU data by fine-tuning a pretrained diffusion model with an acceleration-based second-order loss (L_acc). L_acc enforces consistency in the discrete second-order temporal differences of the generated motion, thereby aligning the diffusion prior with IMU-specific acceleration patterns. We integrate L_acc into the training objective of an existing diffusion model, finetune the model to obtain an IMU-specific motion prior, and evaluate the model with an existing text-to-IMU framework that comprises surface modelling and virtual sensor simulation. We analysed acceleration signal fidelity and differences between synthetic motion representation and actual IMU recordings. As a downstream application, we evaluated Human Activity Recognition (HAR) and compared the classification performance using data of our method with the earlier diffusion model and two additional diffusion model baselines. When we augmented the earlier diffusion model objective with L_acc and continued training, L_acc decreased by 12.7% relative to the original model. The improvements were considerably larger in high-dynamic activities (i.e., running, jumping) compared to low-dynamic activities~(i.e., sitting, standing). In a low-dimensional embedding, the synthetic IMU data produced by our refined model shifts closer to the distribution of real IMU recordings. HAR classification trained exclusively on our refined synthetic IMU data improved performance by 8.7% compared to the earlier diffusion model and by 7.6% over the best-performing comparison diffusion model. We conclude that acceleration-aware diffusion refinement provides an effective approach to align motion generation and IMU synthesis and highlights how flexible deep learning pipelines are for specialising generic text-to-motion priors to sensor-specific tasks.

cs.LG↗

arXiv2024

Space-time deep neural network approximations for high-dimensional partial differential equations

It is one of the most challenging issues in applied mathematics to approximately solve high-dimensional partial differential equations (PDEs) and most of the numerical approximation methods for PDEs in the scientific literature suffer from the so-called curse of dimensionality in the sense that the number of computational operations employed in the corresponding approximation scheme to obtain an approximation precision $\varepsilon>0$ grows exponentially in the PDE dimension and/or the reciprocal of $\varepsilon$. Recently, certain deep learning based approximation methods for PDEs have been proposed and various numerical simulations for such methods suggest that deep neural network (DNN) approximations might have the capacity to indeed overcome the curse of dimensionality in the sense that the number of real parameters used to describe the approximating DNNs grows at most polynomially in both the PDE dimension $d\in\mathbb{N}$ and the reciprocal of the prescribed accuracy $\varepsilon>0$. There are now also a few rigorous results in the scientific literature which substantiate this conjecture by proving that DNNs overcome the curse of dimensionality in approximating solutions of PDEs. Each of these results establishes that DNNs overcome the curse of dimensionality in approximating suitable PDE solutions at a fixed time point $T>0$ and on a compact cube $[a,b]^d$ in space but none of these results provides an answer to the question whether the entire PDE solution on $[0,T]\times [a,b]^d$ can be approximated by DNNs without the curse of dimensionality. It is precisely the subject of this article to overcome this issue. More specifically, the main result of this work in particular proves for every $a\in\mathbb{R}$, $ b\in (a,\infty)$ that solutions of certain Kolmogorov PDEs can be approximated by DNNs on the space-time region $[0,T]\times [a,b]^d$ without the curse of dimensionality.

math.PR↗

arXiv2021

Approximation properties of Residual Neural Networks for Kolmogorov PDEs

In recent years residual neural networks (ResNets) as introduced by [He, K., Zhang, X., Ren, S., and Sun, J., Proceedings of the IEEE conference on computer vision and pattern recognition (2016), 770-778] have become very popular in a large number of applications, including in image classification and segmentation. They provide a new perspective in training very deep neural networks without suffering the vanishing gradient problem. In this article we show that ResNets are able to approximate solutions of Kolmogorov partial differential equations (PDEs) with constant diffusion and possibly nonlinear drift coefficients without suffering the curse of dimensionality, which is to say the number of parameters of the approximating ResNets grows at most polynomially in the reciprocal of the approximation accuracy $\varepsilon > 0$ and the dimension of the considered PDE $d\in\mathbb{N}$. We adapt a proof in [Jentzen, A., Salimova, D., and Welti, T., Commun. Math. Sci. 19, 5 (2021), 1167-1205] - who showed a similar result for feedforward neural networks (FNNs) - to ResNets. In contrast to FNNs, the Euler-Maruyama approximation structure of ResNets simplifies the construction of the approximating ResNets substantially. Moreover, contrary to the above work, in our proof using ResNets does not require the existence of an FNN (or a ResNet) representing the identity map, which enlarges the set of applicable activation functions.

math.NA↗

arXiv2020

Weak error analysis for stochastic gradient descent optimization algorithms

Stochastic gradient descent (SGD) type optimization schemes are fundamental ingredients in a large number of machine learning based algorithms. In particular, SGD type optimization schemes are frequently employed in applications involving natural language processing, object and face recognition, fraud detection, computational advertisement, and numerical approximations of partial differential equations. In mathematical convergence results for SGD type optimization schemes there are usually two types of error criteria studied in the scientific literature, that is, the error in the strong sense and the error with respect to the objective function. In applications one is often not only interested in the size of the error with respect to the objective function but also in the size of the error with respect to a test function which is possibly different from the objective function. The analysis of the size of this error is the subject of this article. In particular, the main result of this article proves under suitable assumptions that the size of this error decays at the same speed as in the special case where the test function coincides with the objective function.

math.NA↗

arXiv2019

A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients

In recent years deep artificial neural networks (DNNs) have been successfully employed in numerical simulations for a multitude of computational problems including, for example, object and face recognition, natural language processing, fraud detection, computational advertisement, and numerical approximations of partial differential equations (PDEs). These numerical simulations indicate that DNNs seem to possess the fundamental flexibility to overcome the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $ \varepsilon > 0 $ and the dimension $ d \in \mathbb{N}$ of the function which the DNN aims to approximate in such computational problems. There is also a large number of rigorous mathematical approximation results for artificial neural networks in the scientific literature but there are only a few special situations where results in the literature can rigorously justify the success of DNNs in high-dimensional function approximation. The key contribution of this paper is to reveal that DNNs do overcome the curse of dimensionality in the numerical approximation of Kolmogorov PDEs with constant diffusion and nonlinear drift coefficients. We prove that the number of parameters used to describe the employed DNN grows at most polynomially in both the PDE dimension $ d \in \mathbb{N}$ and the reciprocal of the prescribed approximation accuracy $ \varepsilon > 0 $. A crucial ingredient in our proof is the fact that the artificial neural network used to approximate the solution of the PDE is indeed a deep artificial neural network with a large number of hidden layers.

math.NA↗

arXiv2019

Deep neural network approximations for Monte Carlo algorithms

Recently, it has been proposed in the literature to employ deep neural networks (DNNs) together with stochastic gradient descent methods to approximate solutions of PDEs. There are also a few results in the literature which prove that DNNs can approximate solutions of certain PDEs without the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially both in the PDE dimension and the reciprocal of the prescribed approximation accuracy. One key argument in most of these results is, first, to use a Monte Carlo approximation scheme which can approximate the solution of the PDE under consideration at a fixed space-time point without the curse of dimensionality and, thereafter, to prove that DNNs are flexible enough to mimic the behaviour of the used approximation scheme. Having this in mind, one could aim for a general abstract result which shows under suitable assumptions that if a certain function can be approximated by any kind of (Monte Carlo) approximation scheme without the curse of dimensionality, then this function can also be approximated with DNNs without the curse of dimensionality. It is a key contribution of this article to make a first step towards this direction. In particular, the main result of this paper, essentially, shows that if a function can be approximated by means of some suitable discrete approximation scheme without the curse of dimensionality and if there exist DNNs which satisfy certain regularity properties and which approximate this discrete approximation scheme without the curse of dimensionality, then the function itself can also be approximated with DNNs without the curse of dimensionality. As an application of this result we establish that solutions of suitable Kolmogorov PDEs can be approximated with DNNs without the curse of dimensionality.

math.NA↗

arXiv2019

Strong and weak divergence of exponential and linear-implicit Euler approximations for stochastic partial differential equations with superlinearly growing nonlinearities

The explicit Euler scheme and similar explicit approximation schemes (such as the Milstein scheme) are known to diverge strongly and numerically weakly in the case of one-dimensional stochastic ordinary differential equations with superlinearly growing nonlinearities. It remained an open question whether such a divergence phenomenon also holds in the case of stochastic partial differential equations with superlinearly growing nonlinearities such as stochastic Allen-Cahn equations. In this work we solve this problem by proving that full-discrete exponential Euler and full-discrete linear-implicit Euler approximations diverge strongly and numerically weakly in the case of stochastic Allen-Cahn equations. This article also contains a short literature overview on existing numerical approximation results for stochastic differential equations with superlinearly growing nonlinearities.

math.NA↗

arXiv2019

Existence, uniqueness, and numerical approximations for stochastic Burgers equations

In this paper we propose an all-in-one statement which includes existence, uniqueness, regularity, and numerical approximations of mild solutions for a class of stochastic partial differential equations (SPDEs) with non-globally monotone nonlinearities. The proof of this result exploits the properties of an existent fully explicit space-time discrete approximation scheme and, in particular, the fact that it satisfies suitable a priori estimates. As a byproduct we obtain almost sure and strong convergence of the approximation scheme to the mild solutions of the considered SPDEs. We conclude by applying the main result of the paper to the stochastic Burgers equations with space-time white noise.

math.PR↗

arXiv2018

Existence and uniqueness properties for solutions of a class of Banach space valued evolution equations

In this note we provide a self-contained proof of an existence and uniqueness result for a class of Banach space valued evolution equations with an additive forcing term. The framework of our abstract result includes, for example, finite dimensional ordinary differential equations (ODEs), semilinear deterministic partial differential equations (PDEs), as well as certain additive noise driven stochastic partial differential equations (SPDEs) as special cases. The framework of our general result assumes somehow mild regularity conditions on the involved semigroup and also allows the involved semigroup operators to be nonlinear. The techniques used in the proofs of our results are essentially well-known in the relevant literature. The contribution of this note is to provide a rather general existence and uniqueness result which covers several situations as special cases and also to provide a self-contained proof for this existence and uniqueness result.

math.CA↗

arXiv2017

Strong convergence for explicit space-time discrete numerical approximation methods for stochastic Burgers equations

In this paper we propose and analyze explicit space-time discrete numerical approximations for additive space-time white noise driven stochastic partial differential equations (SPDEs) with non-globally monotone nonlinearities such as the stochastic Burgers equation with space-time white noise. The main result of this paper proves that the proposed explicit space-time discrete approximation method converges strongly to the solution process of the stochastic Burgers equation with space-time white noise. To the best of our knowledge, the main result of this work is the first result in the literature which establishes strong convergence for a space-time discrete approximation method in the case of the stochastic Burgers equations with space-time white noise.

math.NA↗

arXiv2017

On stochastic differential equations with arbitrarily slow convergence rates for strong approximation in two space dimensions

In the recent article [Jentzen, A., Müller-Gronbach, T., and Yaroslavtseva, L., Commun. Math. Sci., 14(6), 1477--1500, 2016] it has been established that for every arbitrarily slow convergence speed and every natural number $d \in \{4,5,\ldots\}$ there exist $d$-dimensional stochastic differential equations (SDEs) with infinitely often differentiable and globally bounded coefficients such that no approximation method based on finitely many observations of the driving Brownian motion can converge in absolute mean to the solution faster than the given speed of convergence. In this paper we strengthen the above result by proving that this slow convergence phenomena also arises in two ($d=2$) and three ($d=3$) space dimensions.

math.NA↗

arXiv2017

Strong convergence of full-discrete nonlinearity-truncated accelerated exponential Euler-type approximations for stochastic Kuramoto-Sivashinsky equations

This article introduces and analyzes a new explicit, easily implementable, and full discrete accelerated exponential Euler-type approximation scheme for additive space-time white noise driven stochastic partial differential equations (SPDEs) with possibly non-globally monotone nonlinearities such as stochastic Kuramoto-Sivashinsky equations. The main result of this article proves that the proposed approximation scheme converges strongly and numerically weakly to the solution process of such an SPDE. Key ingredients in the proof of our convergence result are a suitable generalized coercivity-type condition, the specific design of the accelerated exponential Euler-type approximation scheme, and an application of Fernique's theorem.

math.PR↗