arXiv subjects

Zhi Zhou

Publications and source records attributed to Zhi Zhou.

At least 55 records · Page 3Linked to original sources

arXiv2025

Numerical analysis of 2D Navier--Stokes equations with nonsmooth initial value in the critical space

This paper addresses the numerical solution of the two-dimensional Navier--Stokes (NS) equations with nonsmooth initial data in the $L^2$ space, which is the critical space for the two-dimensional NS equations to be well-posed. In this case, the solutions of the NS equations exhibit certain singularities at $t=0$, e.g., the $H^s$ norm of the solution blows up as $t\rightarrow 0$ when $s>0$. To date, the best convergence result proved in the literature are first-order accuracy in both time and space for the semi-implicit Euler time-stepping scheme and divergence-free finite elements (even high-order finite elements are used), while numerical results demonstrate that second-order convergence in time and space may be achieved. Therefore, there is still a gap between numerical analysis and numerical computation for the NS equations with $L^2$ initial data. The primary challenge to realizing high-order convergence is the insufficient regularity in the solutions due to the rough initial condition and the nonlinearity of the equations. In this work, we propose a fully discrete numerical scheme that utilizes the Taylor--Hood or Stokes-MINI finite element method for spatial discretization and an implicit-explicit Runge--Kutta time-stepping method in conjunction with graded stepsizes. By employing discrete semigroup techniques, sharp regularity estimates, negative norm estimates and the $L^2$ projection onto the divergence-free Raviart--Thomas element space, we prove that the proposed scheme attains second-order convergence in both space and time. Numerical examples are presented to support the theoretical analysis. In particular, the convergence in space is at most second order even higher-order finite elements are used. This shows the sharpness of the convergence order proved in this article.

math.NA↗

arXiv2025

VCSearch: Bridging the Gap Between Well-Defined and Ill-Defined Problems in Mathematical Reasoning

Large language models (LLMs) have demonstrated impressive performance on reasoning tasks, including mathematical reasoning. However, the current evaluation mostly focuses on carefully constructed benchmarks and neglects the consideration of real-world reasoning problems that present missing or contradictory conditions, known as ill-defined problems. To further study this problem, we develop a largescale benchmark called Problems with Missing and Contradictory conditions (PMC) containing over 5,000 validated ill-defined mathematical problems. Our preliminary experiments through PMC reveal two challenges about existing methods: (1) traditional methods exhibit a trade-off between solving accuracy and rejection capabilities, and (2) formal methods struggle with modeling complex problems. To address these challenges, We develop Variable-Constraint Search (VCSEARCH), a trainingfree framework that leverages formal language to detect ill-defined problems, where a variableconstraint pair search strategy is incorporated to improve the modeling capability of formal language. Extensive experiments demonstrate that VCSEARCH improves the accuracy of identifying unsolvable problems by at least 12% across different LLMs, thus achieving stronger robust mathematical reasoning ability.

cs.AI↗

arXiv2025

CGI: Identifying Conditional Generative Models with Example Images

Generative models have achieved remarkable performance recently, and thus model hubs have emerged. Existing model hubs typically assume basic text matching is sufficient to search for models. However, in reality, due to different abstractions and the large number of models in model hubs, it is not easy for users to review model descriptions and example images, choosing which model best meets their needs. Therefore, it is necessary to describe model functionality wisely so that future users can efficiently search for the most suitable model for their needs. Efforts to address this issue remain limited. In this paper, we propose Conditional Generative Model Identification (CGI), which aims to provide an effective way to identify the most suitable model using user-provided example images rather than requiring users to manually review a large number of models with example images. To address this problem, we propose the PromptBased Model Identification (PMI) , which can adequately describe model functionality and precisely match requirements with specifications. To evaluate PMI approach and promote related research, we provide a benchmark comprising 65 models and 9100 identification tasks. Extensive experimental and human evaluation results demonstrate that PMI is effective. For instance, 92% of models are correctly identified with significantly better FID scores when four example images are provided.

cs.CV↗

arXiv2025

Detecting Scarce and Sparse Anomalous: Solving Dual Imbalance in Multi-Instance Learning

In real-world applications, it is highly challenging to detect anomalous samples with extremely sparse anomalies, as they are highly similar to and thus easily confused with normal samples. Moreover, the number of anomalous samples is inherently scarce. This results in a dual imbalance Multi-Instance Learning (MIL) problem, manifesting at both the macro and micro levels. To address this "needle-in-a-haystack problem", we find that MIL problem can be reformulated as a fine-grained PU learning problem. This allows us to address the imbalance issue in an unbiased manner using micro-level balancing mechanisms. To this end, we propose a novel framework, Balanced Fine-Grained Positive-Unlabeled (BFGPU)-based on rigorous theoretical foundations. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of BFGPU.

stat.ML↗

arXiv2025

FormalML: A Benchmark for Evaluating Formal Subgoal Completion in Machine Learning Theory

Large language models (LLMs) have recently demonstrated remarkable progress in formal theorem proving. Yet their ability to serve as practical assistants for mathematicians, filling in missing steps within complex proofs, remains underexplored. We identify this challenge as the task of subgoal completion, where an LLM must discharge short but nontrivial proof obligations left unresolved in a human-provided sketch. To study this problem, we introduce FormalML, a Lean 4 benchmark built from foundational theories of machine learning. Using a translation tactic that converts procedural proofs into declarative form, we extract 4937 problems spanning optimization and probability inequalities, with varying levels of difficulty. FormalML is the first subgoal completion benchmark to combine premise retrieval and complex research-level contexts. Evaluation of state-of-the-art provers highlights persistent limitations in accuracy and efficiency, underscoring the need for more capable LLM-based theorem provers for effective subgoal completion,

cs.CL↗

arXiv2025

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism

The significant resource demands in LLM serving prompts production clusters to fully utilize heterogeneous hardware by partitioning LLM models across a mix of high-end and low-end GPUs. However, existing parallelization approaches often struggle to scale efficiently in heterogeneous environments due to their coarse-grained and static parallelization strategies. In this paper, we introduce Hetis, a new LLM system tailored for heterogeneous GPU clusters. Hetis addresses two critical challenges: (1) memory inefficiency caused by the mismatch between memory capacity and computational power in heterogeneous devices, and (2) computational inefficiency arising from performance gaps across different LLM modules. To tackle these issues, Hetis employs a fine-grained and dynamic parallelism design. Specifically, it selectively parallelizes compute-intensive operations to reduce latency and dynamically distributes Attention computations to low-end GPUs at a head granularity, leveraging the distinct characteristics of each module. Additionally, Hetis features an online load dispatching policy that continuously optimizes serving performance by carefully balancing network latency, computational load, and memory intensity. Evaluation results demonstrate that Hetis can improve serving throughput by up to $2.25\times$ and reduce latency by $1.49\times$ compared to existing systems.

cs.DC↗

arXiv2025

Hölder Stable Recovery of the Source in Space-Time Fractional Wave Equations

We study the recovery of a spatially dependent source in a one-dimensional space-time fractional wave equation using boundary measurement data collected at a single endpoint. The main challenge arises from the fact that the eigenfunctions of the Dirichlet eigenvalue problem do not form an orthogonal system, due to the presence of a fractional derivative in space. To address this difficulty, we introduce a bi-orthogonal basis for the Mittag-Leffler functions and use it to establish uniqueness and Hölder-type stability results, provided the measurement time is sufficiently large. A Tikhonov regularization method is then employed to numerically solve the inverse source problem. Several numerical examples are presented to demonstrate the accuracy and efficiency of the proposed method and to validate our theoretical findings.

math.AP↗

arXiv2025

Correction of weighted and shifted seven-step BDF for parabolic equations with nonsmooth data

It is well known that the seven-step backward difference formula (BDF) is unstable for the parabolic equations, since it is not even zero-stable. However, a linear combination of two non zero-stable schemes, namely the seven-step BDF and its shifted counterpart, can yield $A(α)$ stable. Based on this observation, the authors [Akrivis, Chen, and Yu, IMA J. Numer. Anal., DOI:10.1093/imanum/drae089] propose the weighted and shifted seven-step BDF methods for the parabolic equations, which stability regions are larger than the standard BDF. Nonetheless, this approach is not directly applicable for the parabolic equations with nonsmooth data, which may suffer from severe order reduction. This motivates us to design proper correction time-stepping schemes to restore the desired $k$th-order convergence rate of the $k$-step weighted and shifted BDF ($k\leq 7$) convolution quadrature for the parabolic problems. We prove that the desired $k$th-order convergence can be recovered even if the source term is discontinuous and the initial value is nonsmooth data. Numerical experiments illustrate the theoretical results.

math.NA↗

arXiv2025

Numerical Method for Space-Time Fractional Diffusion: A Stochastic Approach

In this paper, we develop and analyze a stochastic algorithm for solving space-time fractional diffusion models, which are widely used to describe anomalous diffusion dynamics. These models pose substantial numerical challenges due to the memory effect of the time-fractional derivative and the nonlocal nature of the spatial fractional Laplacian and the, leading to significant computational costs and storage demands, particularly in high-dimensional settings. To overcome these difficulties, we propose a Monte Carlo method based on the Feynman--Kac formula for space-time fractional models. The novel algorithm combines the simulation of the monotone path of a stable subordinator in time with the ``walk-on-spheres'' method that efficiently simulates the stable Levy jumping process in space. We rigorously derive error bounds for the proposed scheme, explicitly expressed in terms of the number of simulation paths and the time step size. Numerical experiments confirm the theoretical error bounds and demonstrate the computational efficiency of the method, particularly in domains with complex geometries or high-dimensional spaces. Furthermore, both theoretical and numerical results emphasize the robustness of the proposed approach across a range of fractional orders, particularly for small fractional values, a capability often absent in traditional numerical methods.

math.NA↗

arXiv2025

Numerical Analysis of Unsupervised Learning Approaches for Parameter Identification in PDEs

Identifying parameters in partial differential equations (PDEs) represents a very broad class of applied inverse problems. In recent years, several unsupervised learning approaches using (deep) neural networks have been developed to solve PDE parameter identifications. These approaches employ neural networks as ansatz functions to approximate the parameters and / or the states, and have demonstrated impressive empirical performance. In this paper, we provide a comprehensive survey on these unsupervised learning techniques on one model problem, diffusion coefficient identification, from the classical numerical analysis perspective, and outline a general framework for deriving rigorous error bounds on the discrete approximations obtained using the Galerkin finite element method, hybrid method and deep neural networks. Throughout we highlight the crucial role of conditional stability estimates in the error analysis.

math.NA↗

arXiv2025

Conditional Stability and Numerical Reconstruction of a Parabolic Inverse Source Problem Using Carleman Estimates

In this work we develop a new numerical approach for recovering a spatially dependent source component in a standard parabolic equation from partial interior measurements. We establish novel conditional Lipschitz stability and Hölder stability for the inverse problem with and without boundary conditions, respectively, using suitable Carleman estimates. Then we propose a numerical approach for solving the inverse problem using conforming finite element approximations in both time and space. Moreover, by utilizing the conditional stability estimates, we prove rigorous error bounds on the discrete approximation. We present several numerical experiments to illustrate the effectiveness of the approach.

math.NA↗

arXiv2025

Smart Charging Impact Analysis using Clustering Methods and Real-world Distribution Feeders

The anticipated widespread adoption of electric vehicles (EVs) necessitates a critical evaluation of existing power distribution infrastructures, as EV integration imposes additional stress on distribution networks that can lead to component overloading and power quality degradation. Implementing smart charging mechanisms can mitigate these adverse effects and defer or even avoid upgrades. This study assesses the performance of two smart charging strategies - Time of Use (TOU) pricing and Load Balancing (LB) on seven representative real-world feeders identified using k-means clustering. A time series-based steady-state load flow analysis was conducted on these feeders to simulate the impact of EV charging under both strategies across four different EV enrollment scenarios and three representative days to capture seasonal load characteristics. A grid upgrade strategy has been proposed to strengthen the power grid to support EV integration with minimal cost. Results demonstrate that both TOU and LB strategies effectively manage the additional EV load with reduced upgrade requirement and cost to existing infrastructure compared to the case without smart charging strategies and LB outperforms TOU when the customer enrollment levels are high. These findings support the viability of smart charging in facilitating EV integration while maintaining distribution network reliability and reducing investment cost.

eess.SY↗

arXiv2025

Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models

Large Language Models (LLMs) have shown promising results across various tasks, yet their reasoning capabilities remain a fundamental challenge. Developing AI systems with strong reasoning capabilities is regarded as a crucial milestone in the pursuit of Artificial General Intelligence (AGI) and has garnered considerable attention from both academia and industry. Various techniques have been explored to enhance the reasoning capabilities of LLMs, with neuro-symbolic approaches being a particularly promising way. This paper comprehensively reviews recent developments in neuro-symbolic approaches for enhancing LLM reasoning. We first present a formalization of reasoning tasks and give a brief introduction to the neurosymbolic learning paradigm. Then, we discuss neuro-symbolic methods for improving the reasoning capabilities of LLMs from three perspectives: Symbolic->LLM, LLM->Symbolic, and LLM+Symbolic. Finally, we discuss several key challenges and promising future directions. We have also released a GitHub repository including papers and resources related to this survey: https://github.com/LAMDASZ-ML/Awesome-LLM-Reasoning-with-NeSy.

cs.AI↗

arXiv2025

Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts

Large Language Models (LLMs) have demonstrated remarkable capabilities, leading to a significant increase in user demand for LLM services. However, cloud-based LLM services often suffer from high latency, unstable responsiveness, and privacy concerns. Therefore, multiple LLMs are usually deployed at the network edge to boost real-time responsiveness and protect data privacy, particularly for many emerging smart mobile and IoT applications. Given the varying response quality and latency of LLM services, a critical issue is how to route user requests from mobile and IoT devices to an appropriate LLM service (i.e., edge LLM expert) to ensure acceptable quality-of-service (QoS). Existing routing algorithms fail to simultaneously address the heterogeneity of LLM services, the interference among requests, and the dynamic workloads necessary for maintaining long-term stable QoS. To meet these challenges, in this paper we propose a novel deep reinforcement learning (DRL)-based QoS-aware LLM routing framework for sustained high-quality LLM services. Due to the dynamic nature of the global state, we propose a dynamic state abstraction technique to compactly represent global state features with a heterogeneous graph attention network (HAN). Additionally, we introduce an action impact estimator and a tailored reward function to guide the DRL agent in maximizing QoS and preventing latency violations. Extensive experiments on both Poisson and real-world workloads demonstrate that our proposed algorithm significantly improves average QoS and computing resource efficiency compared to existing baselines.

cs.NI↗

arXiv2025

Basic Reading Distillation

Large language models (LLMs) have demonstrated remarkable abilities in various natural language processing areas, but they demand high computation resources which limits their deployment in real-world. Distillation is one technique to solve this problem through either knowledge distillation or task distillation. Both distillation approaches train small models to imitate specific features of LLMs, but they all neglect basic reading education for small models on generic texts that are \emph{unrelated} to downstream tasks. In this paper, we propose basic reading distillation (BRD) which educates a small model to imitate LLMs basic reading behaviors, such as named entity recognition, question raising and answering, on each sentence. After such basic education, we apply the small model on various tasks including language inference benchmarks and BIG-bench tasks. It shows that the small model can outperform or perform comparable to over 20x bigger LLMs. Analysis reveals that BRD effectively influences the probability distribution of the small model, and has orthogonality to either knowledge distillation or task distillation.

cs.CL↗

arXiv2025

Learning-Enhanced Variational Regularization for Electrical Impedance Tomography via Calderón's Method

This paper aims to numerically solve the two-dimensional electrical impedance tomography (EIT) with Cauchy data. This inverse problem is highly challenging due to its severe ill-posed nature and strong nonlinearity, which necessitates appropriate regularization strategies. Choosing a regularization approach that effectively incorporates the \textit{a priori} information of the conductivity distribution (or its contrast) is therefore essential. In this work, we propose a deep learning-based method to capture the \textit{a priori} information about the shape and location of the unknown contrast using Calderón's method. The learned \textit{a priori} information is then used to construct the regularization functional of the variational regularization method for solving the inverse problem. The resulting regularized variational problem for EIT reconstruction is then solved using the Gauss-Newton method. Extensive numerical experiments demonstrate that the proposed inversion algorithm achieves accurate reconstruction results, even in high-contrast cases, and exhibits strong generalization capabilities. Additionally, some stability and convergence analysis of the variational regularization method underscores the importance of incorporating \textit{a priori} information about the support of the unknown contrast.

math.NA↗

arXiv2025

Numerical Approximation and Analysis of the Inverse Robin Problem Using the Kohn-Vogelius Method

In this work, we numerically investigate the inverse Robin problem of recovering a piecewise constant Robin coefficient in an elliptic or parabolic problem from the Cauchy data on a part of the boundary, a problem that commonly arises in applications such as non-destructive corrosion detection. We employ a Kohn-Vogelius type variational functional for the regularized reconstruction, and discretize the resulting optimization problem using the Galerkin finite element method on a graded mesh. We establish rigorous error estimates on the recovered Robin coefficient in terms of the mesh size, temporal step size and noise level. This is achieved by combining the approximation error of the direct problem, a priori estimates on the functional, and suitable conditional stability estimates of the continuous inverse problem. We present several numerical experiments to illustrate the approach and to complement the theoretical findings.

math.NA↗

arXiv2025

TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments

Tabular data is widely utilized in various machine learning tasks. Current tabular learning research predominantly focuses on closed environments, while in real-world applications, open environments are often encountered, where distribution and feature shifts occur, leading to significant degradation in model performance. Previous research has primarily concentrated on mitigating distribution shifts, whereas feature shifts, a distinctive and unexplored challenge of tabular data, have garnered limited attention. To this end, this paper conducts the first comprehensive study on feature shifts in tabular data and introduces the first tabular feature-shift benchmark (TabFSBench). TabFSBench evaluates impacts of four distinct feature-shift scenarios on four tabular model categories across various datasets and assesses the performance of large language models (LLMs) and tabular LLMs in the tabular benchmark for the first time. Our study demonstrates three main observations: (1) most tabular models have the limited applicability in feature-shift scenarios; (2) the shifted feature set importance has a linear relationship with model performance degradation; (3) model performance in closed environments correlates with feature-shift performance. Future research direction is also explored for each observation. Benchmark: https://github.com/LAMDASZ-ML/TabFSBench.

cs.LG↗