arXiv subjects

Junxiong Jia

Publications and source records attributed to Junxiong Jia.

At least 19 recordsLinked to original sources

arXiv2026

Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem

This paper addresses infinite-dimensional Bayesian inference for inverse problem of partial differential equations with model parameters in infinite-dimensional Hilbert space. To effectively incorporate prior information, we propose a novel continuous normalizing flows based infinite-dimensional model. Specifically, by introducing a well-defined neural ordinary differential equation in infinite-dimensional space, a simple reference measure can be transformed into a more complex measure which encodes the prior information. A corresponding theoretical framework is established to ensure the well-posedness of our proposed Bayesian prior in infinite-dimensional space. We also provide training methods of the prior for two distinct data settings, along with two sampling algorithms for the resulting Bayesian posterior. The proposed framework is applied to three representative inverse problems: the simple smooth inverse problem, inverse scattering problem, and the inverse heat conduction problem. Numerical experiments support the theoretical analysis and demonstrate the efficiency of the proposed algorithms.

math.NA↗

arXiv2026

Posterior Consistency for Recovering Initial States in Nonlinear Subdiffusion Equations

We study the Bayesian recovery of the initial state in a semilinear time-fractional subdiffusion equation from noisy random space-time point observations. A rescaled Gaussian prior based on a Whittle--Matérn process is assigned to the unknown initial condition. We prove the $H^{2+κ}$-regularity of the solution when the nonlinearity satisfies a Lipschitz condition in the $H^κ$-norm. We then establish posterior contraction rates for the prediction error in the $L^2$-norm and for the parameter in Sobolev norms. The rates are polynomial in the sample size, with exponent depending on the prior smoothness and the spatial dimension. Moreover, we prove a minimax lower bound by constructing a wavelet-packing set and controlling the Kullback--Leibler divergences.

math.ST↗

arXiv2026

Sequential Monte Carlo with Gaussian Mixture Approximation for Infinite-Dimensional Statistical Inverse Problems

By formulating the inverse problem of partial differential equations (PDEs) as a statistical inference problem, the Bayesian approach provides a general framework for quantifying uncertainties. In the inverse problem of PDEs, parameters are defined on an infinite-dimensional function space, and the PDEs induce a computationally intensive likelihood function. Additionally, sparse data tends to lead to a multi-modal posterior. These features make it difficult to apply existing sequential Monte Carlo (SMC) algorithms. To overcome these difficulties, we propose new conditions for the likelihood functions, construct a Gaussian mixture based preconditioned Crank-Nicolson transition kernel, and demonstrate the universal approximation property of the infinite-dimensional Gaussian mixture probability measure. By combining these three novel tools, we propose a new SMC algorithm with Gaussian mixture approximation, together with an easy-to-use reduced version. For this new algorithm, we obtain a convergence theorem that allows Gaussian priors, illustrating that the sequential particle filter actually reproduces the true posterior distribution. Furthermore, the proposed new algorithm is rigorously defined on the infinite-dimensional function space, naturally exhibiting the discretization-invariant property. Numerical experiments demonstrate that the reduced version has a strong ability to probe the multi-modality of the posterior, significantly reduces the computational burden, and numerically exhibits the discretization-invariant property (important for large-scale problems).

math.NA↗

arXiv2026

Functional normalizing flow for statistical inverse problems of partial differential equations

Inverse problems of partial differential equations are ubiquitous across various scientific disciplines and can be formulated as statistical inference problems using Bayes' theorem. To address large-scale problems, it is crucial to develop discretization-invariant algorithms, which can be achieved by formulating methods directly in infinite-dimensional space. We propose a novel normalizing flow based infinite-dimensional variational inference method (NF-iVI) to extract posterior information efficiently. Specifically, by introducing well-defined transformations, the prior in Bayes' formula is transformed into post-transformed measures that approximate the posterior. To circumvent the issue of mutually singular probability measures, we formulate general conditions for the employed transformations. As guiding principles, these conditions yield four concrete transformations. Additionally, to minimize computational demands, we have developed a conditional normalizing flow variant, termed CNF-iVI, which is adapt at processing measurement data of varying dimensions while requiring minimal computational resources. We apply the proposed algorithms to three typical inverse problems governed by the simple smooth equation, the steady-state Darcy flow equation, and the electric impedance tomography. Numerical results confirm our theoretical findings, illustrate the efficiency of our algorithms, and verify the discretization-invariant property.

math.NA↗

arXiv2026

A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws

Emergent intelligence have played a major role in the modern AI development. While existing studies primarily rely on empirical observations to characterize this phenomenon, a rigorous theoretical framework remains underexplored. This study attempts to develop a mathematical approach to formalize emergent intelligence from the perspective of limit theory. Specifically, we introduce a performance function E(N, P, K), dependent on data size N, model size P and training steps K, to quantify intelligence behavior. We posit that intelligence emerges as a transition from finite to effectively infinite knowledge, and thus recast emergent intelligence as existence of the limit $\lim_{N,P,K \to \infty} \mathcal{E}(N,P,K)$, with emergent abilities corresponding to the limiting behavior. This limit theory helps reveal that emergent intelligence originates from the existence of a parameter-limit architecture (referred to as the limit architecture), and that emergent intelligence rationally corresponds to the learning behavior of this limit system. By introducing tools from nonlinear Lipschitz operator theory, we prove that the necessary and sufficient conditions for existence of the limit architecture. Furthermore, we derive the scaling law of foundation models by leveraging tools of Lipschitz operator and covering number. Theoretical results show that: 1) emergent intelligence is governed by three key factors-training steps, data size and the model architecture, where the properties of basic blocks play a crucial role in constructing foundation models; 2) the critical condition Lip(T)=1 for emergent intelligence provides theoretical support for existing findings. 3) emergent intelligence is determined by an infinite-dimensional system, yet can be effectively realized in practice through a finite-dimensional architecture. Our empirical results corroborate these theoretical findings.

cs.LG↗

arXiv2026

Consistency of Variational Inference for Nonlinear Inverse Problems of Partial Differential Equations

We investigate the convergence rates of variational posterior distributions for statistical inverse problems involving nonlinear partial differential equations (PDEs). Departing from exact Bayesian inference, variational inference transforms the inference problem into an optimization problem by introducing variational sets. Based on a modified ``prior mass and testing'' framework, we propose general conditions for three categories of inverse problems: mildly ill-posed, severely ill-posed, and those with unknown model parameters. Concentrating on the variational sets comprising the restricted Gaussian or widely utilized Gaussian mean-field families, we demonstrate that for all three categories, the convergence rate can be decomposed into a true distribution term and a variational approximation term. Moreover, we illustrate that the true distribution term dominates the convergence rates, thereby substantiating the effectiveness of the variational inference method for inverse problems of PDEs. As specific examples, we examine a collection of non-linear inverse problems, including the Darcy flow problem, the inverse potential problem for a subdiffusion equation, and the inverse medium scattering problem. Besides, we show that our convergence rates are minimax optimal for these inverse problems.

math.ST↗

arXiv2026

Consistency of variational inference for Besov priors in non-linear inverse problems

This study investigates the variational posterior convergence rates of inverse problems for partial differential equations (PDEs) with parameters in Besov spaces $B_{pp}^α$ ($p \geq 1$) which are modeled naturally in a Bayesian manner using Besov priors constructed via random wavelet expansions with $p$-exponentially distributed coefficients. Departing from exact Bayesian inference, variational inference transforms the inference problem into an optimization problem by introducing variational sets. Building on a refined ``prior mass and testing'' framework, we derive general conditions on PDE operators and guarantee that variational posteriors achieve convergence rates matching those of the exact posterior under widely adopted variational families (Besov-type measures or mean-field families). Moreover, our results achieve minimax-optimal rates over $B^α_{pp}$ classes, significantly outperforming the suboptimal rates of Gaussian priors (by a polynomial factor). As specific examples, two typical nonlinear inverse problems, the Darcy flow problems and the inverse potential problem for a subdiffusion equation, are investigated to validate our theory. Besides, we show that our convergence rates of ``prediction'' loss for these ``PDE-constrained regression problems'' are minimax optimal.

math.ST↗

arXiv2026

Stochastic gradient descent based variational inference for infinite-dimensional inverse problems

This paper introduces two variational inference approaches for infinite-dimensional inverse problems, developed through gradient descent with a constant learning rate. The proposed methods enable efficient approximate sampling from the target posterior distribution using a constant-rate stochastic gradient descent (cSGD) iteration. Specifically, we introduce a randomization strategy that incorporates stochastic gradient noise, allowing the cSGD iteration to be viewed as a discrete-time process. This transformation establishes key relationships between the covariance operators of the approximate and true posterior distributions, thereby validating cSGD as a variational inference method. We also investigate the regularization properties of the cSGD iteration and provide a theoretical analysis of the discretization error between the approximated posterior mean and the true background function. Building on this framework, we develop a preconditioned version of cSGD to further improve sampling efficiency. Finally, we apply the proposed methods to two practical inverse problems: one governed by a simple smooth equation and the other by the steady-state Darcy flow equation. Numerical results confirm our theoretical findings and compare the sampling performance of the two approaches for solving linear and non-linear inverse problems.

math.NA↗

arXiv2026

Nonparametric Prior Learning in Differential Equation Modeling

This paper addresses Bayesian inference related to partial differential equations (PDEs), particularly nonparametric regression constrained by PDEs. To effectively encode prior information, we propose a novel framework that learns a prediction function of the prior distribution from historical training datasets. We introduce hyper-prior and hyper-posterior distributions and derive a generalization error estimate, which accommodates data-dependent priors by extending the concept of differential privacy. Some mild conditions are given to validate the error estimate, where various typical PDEs such as diffusion and Darcy flow equations can be integrated. We thus formulate an infinite-dimensional optimization problem to obtain the point estimate of the hyper-posterior. Numerical examples demonstrate the performance of our proposed method in learning the prediction function of priors.

math.ST↗

arXiv2023

Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations

This paper proposes a non-centered parameterization based infinite-dimensional mean-field variational inference (NCP-iMFVI) approach for solving the hierarchical Bayesian inverse problems. This method can generate available estimates from the approximated posterior distribution efficiently. To avoid the mutually singular obstacle that occurred in the infinite-dimensional hierarchical approach, we propose a rigorous theory of the non-centered variational Bayesian approach. Since the non-centered parameterization weakens the connection between the parameter and the hyper-parameter, we can introduce the hyper-parameter to all terms of the eigendecomposition of the prior covariance operator. We also show the relationships between the NCP-iMFVI and infinite-dimensional hierarchical approaches with centered parameterization. The proposed algorithm is applied to three inverse problems governed by the simple smooth equation, the Helmholtz equation, and the steady-state Darcy flow equation. Numerical results confirm our theoretical findings, illustrate the efficiency of solving the iMFVI problem formulated by large-scale linear and nonlinear statistical inverse problems, and verify the mesh-independent property.

math.NA↗

arXiv2023

Variational Inverting Network for Statistical Inverse Problems of Partial Differential Equations

To quantify uncertainties in inverse problems of partial differential equations (PDEs), we formulate them into statistical inference problems using Bayes' formula. Recently, well-justified infinite-dimensional Bayesian analysis methods have been developed to construct dimension-independent algorithms. However, there are three challenges for these infinite-dimensional Bayesian methods: prior measures usually act as regularizers and are not able to incorporate prior information efficiently; complex noises, such as more practical non-i.i.d. distributed noises, are rarely considered; and time-consuming forward PDE solvers are needed to estimate posterior statistical quantities. To address these issues, an infinite-dimensional inference framework has been proposed based on the infinite-dimensional variational inference method and deep generative models. Specifically, by introducing some measure equivalence assumptions, we derive the evidence lower bound in the infinite-dimensional setting and provide possible parametric strategies that yield a general inference framework called the Variational Inverting Network (VINet). This inference framework can encode prior and noise information from learning examples. In addition, relying on the power of deep neural networks, the posterior mean and variance can be efficiently and explicitly generated in the inference stage. In numerical experiments, we design specific network structures that yield a computable VINet from the general inference framework. Numerical examples of linear inverse problems of an elliptic equation and the Helmholtz equation are presented to illustrate the effectiveness of the proposed inference framework.

math.NA↗

arXiv2022

Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems

In this paper, we propose an infinite-dimensional version of the Stein variational gradient descent (iSVGD) method for solving Bayesian inverse problems. The method can generate approximate samples from posteriors efficiently. Based on the concepts of operator-valued kernels and vector-valued reproducing kernel Hilbert spaces, a rigorous definition is given for the infinite-dimensional objects, e.g., the Stein operator, which are proved to be the limit of finite-dimensional ones. Moreover, a more efficient iSVGD with preconditioning operators is constructed by generalizing the change of variables formula and introducing a regularity parameter. The proposed algorithms are applied to an inverse problem of the steady state Darcy flow equation. Numerical results confirm our theoretical findings and demonstrate the potential applications of the proposed approach in the posterior sampling of large-scale nonlinear statistical inverse problems.

math.NA↗

arXiv2020

Variational Bayes' method for functions with applications to some inverse problems

Bayesian approach, as a useful tool for quantifying uncertainties, has been widely used for solving inverse problems of partial differential equations (PDEs). One of the key difficulties for employing Bayesian approach for the issue is how to extract information from the posterior probability measure. Variational Bayes' method (VBM) is firstly and broadly studied in the field of machine learning, which has the ability to extract posterior information approximately by using much lower computational resources compared with the conventional sampling type methods. In this paper, we generalize the usual finite-dimensional VBM to infinite-dimensional space, which makes the usage of VBM for inverse problems of PDEs rigorously. We further establish general infinite-dimensional mean-field approximate theory, and apply this theory to abstract linear inverse problems with Gaussian and Laplace noise assumptions. The results on some numerical examples substantiate the effectiveness of the proposed approach.

math.NA↗

arXiv2020

Posterior contraction for empirical Bayesian approach to inverse problems under non-diagonal assumption

We investigate an empirical Bayesian nonparametric approach to a family of linear inverse problems with Gaussian prior and Gaussian noise. We consider a class of Gaussian prior probability measures with covariance operator indexed by a hyperparameter that quantifies regularity. By introducing two auxiliary problems, we construct an empirical Bayes method and prove that this method can automatically select the hyperparameter. In addition, we show that this adaptive Bayes procedure provides optimal contraction rates up to a slowly varying term and an arbitrarily small constant, without knowledge about the regularity index. Our method needs not the prior covariance, noise covariance and forward operator have a common basis in their singular value decomposition, enlarging the application range compared with the existing results.

math.ST↗

arXiv2019

An adaptive augmented regularization method and its applications

Regularization method and Bayesian inverse method are two dominating ways for solving inverse problems generated from various fields, e.g., seismic exploration and medical imaging. The two methods are related with each other by the MAP estimates of posterior probability distributions. Considering this connection, we construct a prior probability distribution with several hyper-parameters and provide the relevant Bayes' formula, then we propose a corresponding adaptive augmented regularization model (AARM). According to the measured data, the proposed AARM can adjust its form to various regularization models at each discrete point of the estimated function, which makes the characterization of local smooth properties of the estimated function possible. By proposing a modified Bregman iterative algorithm, we construct an alternate iterative algorithm to solve the AARM efficiently. In the end, we provide some numerical examples which clearly indicate that the proposed AARM can generates a favorable result for some examples compared with several Tikhonov and Total-Variation regularization models.

math.NA↗

arXiv2019

Recursive linearization method for inverse medium scattering problems with complex mixture Gaussian error learning

This paper is concerned with the modeling errors appeared in the numerical methods of inverse medium scattering problems (IMSP). Optimization based iterative methods are wildly employed to solve IMSP, which are computationally intensive due to a series of Helmholtz equations need to be solved numerically. Hence, rough approximations of Helmholtz equations can significantly speed up the iterative procedure. However, rough approximations will lead to instability and inaccurate estimations. Using the Bayesian inverse methods, we incorporate the modelling errors brought by the rough approximations. Modelling errors are assumed to be some complex Gaussian mixture (CGM) random variables, and in addition, well-posedness of IMSP in the statistical sense has been established by extending the general theory to involve CGM noise. Then, we generalize the real valued expectation-maximization (EM) algorithm used in the machine learning community to our complex valued case to learn parameters in the CGM distribution. Based on these preparations, we generalize the recursive linearization method (RLM) to a new iterative method named as Gaussian mixture recursive linearization method (GMRLM) which takes modelling errors into account. Finally, we provide two numerical examples to illustrate the effectiveness of the proposed method.

math.NA↗

arXiv2018

A New Nonconvex Strategy to Affine Matrix Rank Minimization Problem

The affine matrix rank minimization (AMRM) problem is to find a matrix of minimum rank that satisfies a given linear system constraint. It has many applications in some important areas such as control, recommender systems, matrix completion and network localization. However, the problem (AMRM) is NP-hard in general due to the combinational nature of the matrix rank function. There are many alternative functions have been proposed to substitute the matrix rank function, which lead to many corresponding alternative minimization problems solved efficiently by some popular convex or nonconvex optimization algorithms. In this paper, we propose a new nonconvex function, namely, $TL_α^ε$ function (with $0\leqα<1$ and $ε>0$), to approximate the rank function, and translate the NP-hard problem (AMRM) into the $TL_{p}^ε$ function affine matrix rank minimization (TLAMRM) problem. Firstly, we study the equivalence of problem (AMRM) and (TLAMRM), and proved that the uniqueness of global minimizer of the problem (TLAMRM) also solves the NP-hard problem (AMRM) if the linear map $\mathcal{A}$ satisfies a restricted isometry property (RIP). Secondly, an iterative thresholding algorithm is proposed to solve the regularization problem (RTLAMRM) for all $0\leqα<1$ and $ε>0$. At last, some numerical results on low-rank matrix completion problems illustrated that our algorithm is able to recover a low-rank matrix, and the extensive numerical on image inpainting problems shown that our algorithm performs the best in finding a low-rank image compared with some state-of-art methods.

math.OC↗

arXiv2018

Generalized singular value thresholding operator to affine matrix rank minimization problem

It is well known that the affine matrix rank minimization problem is NP-hard and all known algorithms for exactly solving it are doubly exponential in theory and in practice due to the combinational nature of the rank function. In this paper, a generalized singular value thresholding operator is generated to solve the affine matrix rank minimization problem. Numerical experiments show that our algorithm performs effectively in finding a low-rank matrix compared with some state-of-art methods.

math.OC↗