arXiv Science⌕ Search

arXiv subjects

Yuwei Fan

Publications and source records attributed to Yuwei Fan.

At least 37 records · Page 2Linked to original sources

An improved Shifted CholeskyQR based on columns

Among all the deterministic CholeskyQR-type algorithms, Shifted CholeskyQR3 is specifically designed to address the QR factorization of ill-conditioned matrices. This algorithm introduces a shift parameter $s$ to prevent failure during the initial Cholesky factorization step, making the choice of this parameter critical for the algorithm's effectiveness. Our goal is to identify a smaller $s$ compared to the traditional selection based on $\norm{X}_{2}$. In this research, we propose a new definition for the input matrix $X$ called $[X]_{g}$, which is based on the column properties of $X$. $[X]_{g}$ allows us to obtain a reduced shift parameter $s$ for the Shifted CholeskyQR3 algorithm, thereby improving the sufficient condition of $κ_{2}(X)$ for this method. We provide rigorous proofs of orthogonality and residuals for the improved algorithm using our proposed $s$. Numerical experiments confirm the enhanced numerical stability of orthogonality and residuals with the reduced $s$. We find that Shifted CholeskyQR3 can effectively handle ill-conditioned $X$ with a larger $κ_{2}(X)$ when using our reduced $s$ compared to the original $s$. Furthermore, we compare CPU times with other algorithms to assess performance improvements.

math.NA↗

Fine-Tuning Pre-trained Large Time Series Models for Prediction of Wind Turbine SCADA Data

The remarkable achievements of large models in the fields of natural language processing (NLP) and computer vision (CV) have sparked interest in their application to time series forecasting within industrial contexts. This paper explores the application of a pre-trained large time series model, Timer, which was initially trained on a wide range of time series data from multiple domains, in the prediction of Supervisory Control and Data Acquisition (SCADA) data collected from wind turbines. The model was fine-tuned on SCADA datasets sourced from two wind farms, which exhibited differing characteristics, and its accuracy was subsequently evaluated. Additionally, the impact of data volume was studied to evaluate the few-shot ability of the Timer. Finally, an application study on one-turbine fine-tuning for whole-plant prediction was implemented where both few-shot and cross-turbine generalization capacity is required. The results reveal that the pre-trained large model does not consistently outperform other baseline models in terms of prediction accuracy whenever the data is abundant or not, but demonstrates superior performance in the application study. This result underscores the distinctive advantages of the pre-trained large time series model in facilitating swift deployment.

cs.LG↗

Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines

This study explores the application of large language models (LLMs) with callable tools in energy and power engineering domain, focusing on gas path analysis of gas turbines. We developed a dual-agent tool-calling process to integrate expert knowledge, predefined tools, and LLM reasoning. We evaluated various LLMs, including LLama3, Qwen1.5 and GPT. Smaller models struggled with tool usage and parameter extraction, while larger models demonstrated favorable capabilities. All models faced challenges with complex, multi-component problems. Based on the test results, we infer that LLMs with nearly 100 billion parameters could meet professional scenario requirements with fine-tuning and advanced prompt design. Continued development are likely to enhance their accuracy and effectiveness, paving the way for more robust AI-driven solutions.

cs.AI↗

A Novel Randomized XR-Based Preconditioned CholeskyQR Algorithm

CholeskyQR is a simple and fast QR decomposition via Cholesky decomposition, while it has been considered highly sensitive to the condition number. In this paper, we provide a randomized preconditioner framework for CholeskyQR algorithm. Under this framework, two methods (randomized LU-CholeskyQR and randomized QR-CholeskyQR) are proposed and discussed. We prove the proposed preconditioners can effectively reduce the condition number, which is also demonstrated by numerical tests. Abundant numerical tests indicate our methods are more stable and faster than all the existing algorithms and have good scalability.

math.NA↗

A Simple Multiscale Method for Mean Field Games

This paper proposes a multiscale method for solving the numerical solution of mean field games which accelerates the convergence and addresses the problem of determining the initial guess. Starting from an approximate solution at the coarsest level, the method constructs approximations on successively finer grids via alternating sweeping, which not only allows for the use of classical time marching numerical schemes but also enables applications to both local and nonlocal problems. At each level, numerical relaxation is used to stabilize the iterative process. A second-order discretization scheme is derived for higher-order convergence. Numerical examples are provided to demonstrate the efficiency of the proposed method in both local and nonlocal, 1-dimensional and 2-dimensional cases.

math.NA↗

Multi-Level Fine-Tuning: Closing Generalization Gaps in Approximation of Solution Maps under a Limited Budget for Training Data

In scientific machine learning, regression networks have been recently applied to approximate solution maps (e.g., potential-ground state map of Schrödinger equation). In this paper, we aim to reduce the generalization error without spending more time in generating training samples. However, to reduce the generalization error, the regression network needs to be fit on a large number of training samples (e.g., a collection of potential-ground state pairs). The training samples can be produced by running numerical solvers, which takes much time in many applications. In this paper, we aim to reduce the generalization error without spending more time in generating training samples. Inspired by few-shot learning techniques, we develop the Multi-Level Fine-Tuning algorithm by introducing levels of training: we first train the regression network on samples generated at the coarsest grid and then successively fine-tune the network on samples generated at finer grids. Within the same amount of time, numerical solvers generate more samples on coarse grids than on fine grids. We demonstrate a significant reduction of generalization error in numerical experiments on challenging problems with oscillations, discontinuities, or rough coefficients. Further analysis can be conducted in the Neural Tangent Kernel regime and we provide practical estimators to the generalization error. The number of training samples at different levels can be optimized for the smallest estimated generalization error under the constraint of budget for training data. The optimized distribution of budget over levels provides practical guidance with theoretical insight as in the celebrated Multi-Level Monte Carlo algorithm.

math.NA↗

Hyperbolic Model Reduction for Kinetic Equations

We make a brief historical review to the moment model reduction to the kinetic equations, particularly the Grad's moment method for Boltzmann equation. The focus is on the hyperbolicity of the reduced model, which is essential to the existence of its classical solution as a Cauchy problem. The theory of the framework we developed in last years is then introduced, which may preserve the hyperbolic nature of the kinetic equations with high universality. Some lastest progress on the comparison between models with/without hyperbolicity is presented to validate the hyperbolic moment models for rarefied gases.

cond-mat.stat-mech↗

A Nonlinear Hyperbolic Model for Radiative Transfer Equation in Slab Geometry

Linear models for the radiative transfer equation have been well developed, while nonlinear models are seldom investigated even for slab geometry due to some essential difficulties. We have proposed a moment model in MPN for slab geometry which combines the ideas of the classical PN and MN model. Though the model is far from perfect, it was demonstrated to be quite efficient in numerically approximating the solution of the radiative transfer equation, that we are motivated to further improve this model. Consequently we propose in this paper a new model following the chartmap in MPN with some significant theoretic progresses. The new model is derived with global hyperbolicity, and meanwhile some necessary physical properties are preserved. We give a complete analysis to the characteristic structure and propose a numerical scheme for the new model. Numerical examples are presented to demonstrate the numerical performance of the new model.

math.NA↗

Meta-learning Pseudo-differential Operators with Deep Neural Networks

This paper introduces a meta-learning approach for parameterized pseudo-differential operators with deep neural networks. With the help of the nonstandard wavelet form, the pseudo-differential operators can be approximated in a compressed form with a collection of vectors. The nonlinear map from the parameter to this collection of vectors and the wavelet transform are learned together from a small number of matrix-vector multiplications of the pseudo-differential operator. Numerical results for Green's functions of elliptic partial differential equations and the radiative transfer equations demonstrate the efficiency and accuracy of the proposed approach.

math.NA↗

Solving Inverse Wave Scattering with Deep Learning

This paper proposes a neural network approach for solving two classical problems in the two-dimensional inverse wave scattering: far field pattern problem and seismic imaging. The mathematical problem of inverse wave scattering is to recover the scatterer field of a medium based on the boundary measurement of the scattered wave from the medium, which is high-dimensional and nonlinear. For the far field pattern problem under the circular experimental setup, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this and filtered back-projection, we propose an effective neural network architecture for the inverse map using the recently introduced BCR-Net along with the standard convolution layers. Analogously for the seismic imaging problem, we propose a similar neural network architecture under the rectangular domain setup with a depth-dependent background velocity. Numerical results demonstrate the efficiency of the proposed neural networks.

physics.comp-ph↗

Solving Traveltime Tomography with Deep Learning

This paper introduces a neural network approach for solving two-dimensional traveltime tomography (TT) problems based on the eikonal equation. The mathematical problem of TT is to recover the slowness field of a medium based on the boundary measurement of the traveltimes of waves going through the medium. This inverse map is high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this and filtered back-projection, we propose an effective neural network architecture for the inverse map using the recently proposed BCR-Net, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks.

math.NA↗

Coarse-Proxy Reduced Basis Methods for Integral Equations

In this paper, we introduce a new reduced basis methodology for accelerating the computation of large parameterized systems of high-fidelity integral equations. Core to our methodology is the use of coarse-proxy models (i.e., lower resolution variants of the underlying high-fidelity equations) to identify important samples in the parameter space from which a high quality reduced basis is then constructed. Unlike the more traditional POD or greedy methods for reduced basis construction, our methodology has the benefit of being both easy to implement and embarrassingly parallel. We apply our methodology to the under-served area of integral equations, where the density of the underlying integral operators has traditionally made reduced basis methods difficult to apply. To handle this difficulty, we introduce an operator interpolation technique, based on random sub-sampling, that is aimed specifically at integral operators. To demonstrate the effectiveness of our techniques, we present two numerical case studies, based on the Radiative Transport Equation and a boundary integral formation of the Laplace Equation respectively, where our methodology provides a significant improvement in performance over the underlying high-fidelity models for a wide range of error tolerances. Moreover, we demonstrate that for these problems, as the coarse-proxy selection threshold is made more aggressive, the approximation error of our method decreases at an approximately linear rate.

math.NA↗

A multiscale neural network based on hierarchical matrices

In this work we introduce a new multiscale artificial neural network based on the structure of $\mathcal{H}$-matrices. This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale. Numerical results indicate that the network is able to efficiently approximate discrete nonlinear maps obtained from discretized nonlinear partial differential equations, such as those arising from nonlinear Schrödinger equations and the Kohn-Sham density functional theory.

math.NA↗

Solving Electrical Impedance Tomography with Deep Learning

This paper introduces a new approach for solving electrical impedance tomography (EIT) problems using deep neural networks. The mathematical problem of EIT is to invert the electrical conductivity from the Dirichlet-to-Neumann (DtN) map. Both the forward map from the electrical conductivity to the DtN map and the inverse map are high-dimensional and nonlinear. Motivated by the linear perturbative analysis of the forward map and based on a numerically low-rank property, we propose compact neural network architectures for the forward and inverse maps for both 2D and 3D problems. Numerical results demonstrate the efficiency of the proposed neural networks.

physics.comp-ph↗

Solving Optical Tomography with Deep Learning

This paper presents a neural network approach for solving two-dimensional optical tomography (OT) problems based on the radiative transfer equation. The mathematical problem of OT is to recover the optical properties of an object based on the albedo operator that is accessible from boundary measurements. Both the forward map from the optical properties to the albedo operator and the inverse map are high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this, we propose effective neural network architectures for the forward and inverse maps based on convolution layers, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks.

physics.comp-ph↗

A multiscale neural network based on hierarchical nested bases

In recent years, deep learning has led to impressive results in many fields. In this paper, we introduce a multi-scale artificial neural network for high-dimensional non-linear maps based on the idea of hierarchical nested bases in the fast multipole method and the $\mathcal{H}^2$-matrices. This approach allows us to efficiently approximate discretized nonlinear maps arising from partial differential equations or integral equations. It also naturally extends our recent work based on the generalization of hierarchical matrices [Fan et al. arXiv:1807.01883] but with a reduced number of parameters. In particular, the number of parameters of the neural network grows linearly with the dimension of the parameter space of the discretized PDE. We demonstrate the properties of the architecture by approximating the solution maps of non-linear Schr{ö}dinger equation, the radiative transfer equation, and the Kohn-Sham map.

math.NA↗

A Nonlinear Moment Model for Radiative Transfer Equation in Slab Geometry

This paper is concerned with the approximation of the radiative transfer equation for a grey medium in the slab geometry by the moment method. We develop a novel moment model inspired by the classical $P_N$ model and $M_N$ model. The new model takes the ansatz of the $M_1$ model as the weight function and follows the primary idea of the $P_N$ model to approximate the specific intensity by expanding it around the weight function in terms of orthogonal polynomials. The weight function uses the information of the first two moments, which brings the new model the capability to approximate an anisotropic distribution. Mathematical properties of the moment model are investigated, and particularly the hyperbolicity and the characteristic structure of the Riemann problem of the model with three moments are studied in detail. Some numerical simulations demonstrate its numerical efficiency and show its superior in comparison to the $P_N$ model.

physics.class-ph↗

Filtered Hyperbolic Moment Method for the Vlasov Equation

In this paper, we investigate the effect of the filter for the hyperbolic moment equations(HME) [15] of the Vlasov-Poisson equations and propose a novel quasi time-consistent filter to suppress the numerical recurrence effect. By taking properties of HME into consideration, the filter preserves a lot of physical properties of HME, including Galilean invariance and the conservation of mass, momentum and energy. We present two viewpoints, collisional viewpoint and dissipative viewpoint, to dissect the filter, and show that the filtered hyperbolic moment method can be treated as a solver of Vlasov equation. Numerical simulations of the linear Landau damping and two stream instability are tested to demonstrate the effectiveness of the filter in restraining recurrence arising from particle streaming. Both the analysis and the numerical results indicate that the filtered HME can capture the evolution of the Vlasov equation, even when phase mixing and filamentation are dominant.

math.NA↗