arXiv subjects

Shuai Lu

Publications and source records attributed to Shuai Lu.

At least 73 records · Page 4Linked to original sources

arXiv2025

On the Data-Driven Modeling of Price-Responsive Flexible Loads: Formulation and Algorithm

The flexible loads in power systems, such as interruptible and transferable loads, are critical flexibility resources for mitigating power imbalances. Despite their potential, accurate modeling of these loads is a challenging work and has not received enough attention, limiting their integration into operational frameworks. To bridge this gap, this paper develops a data-driven identification theory and algorithm for price-responsive flexible loads (PRFLs). First, we introduce PRFL models that capture both static and dynamic decision mechanisms governing their response to electricity price variations. Second, We develop a data-driven identification framework that explicitly incorporates forecast and measurement errors. Particularly, we give a theoretical analysis to quantify the statistical impact of such noise on parameter estimation. Third, leveraging the bilevel structure of the identification problem, we propose a Bayesian optimization-based algorithm that features the scalability to large sample sizes and the ability to offer posterior differentiability certificates as byproducts. Numerical tests demonstrate the effectiveness and superiority of the proposed approach.

eess.SY↗

arXiv2025

Dynamic Energy Flow Analysis of Integrated Electricity and Gas Systems: A Semi-Analytical Approach

Ensuring the safe and reliable operation of integrated electricity and gas systems (IEGS) requires dynamic energy flow (DEF) simulation tools that achieve high accuracy and computational efficiency. However, the inherent strong nonlinearity of gas dynamics and its bidirectional coupling with power grids impose significant challenges on conventional numerical algorithms, particularly in computational efficiency and accuracy. Considering this, we propose a novel non-iterative semi-analytical algorithm based on differential transformation (DT) for DEF simulation of IEGS. First, we introduce a semi-discrete difference method to convert the partial differential algebraic equations of the DEF model into ordinary differential algebraic equations to resort to the DT. Particularly, by employing spatial central difference and numerical boundary extrapolation, we effectively avoid the singularity issue of the DT coefficient matrix. Second, we propose a DT-based semi-analytical solution method, which can yield the solution of the DEF model by recursion. Finally, simulation results demonstrate the superiority of the proposed method.

eess.SY↗

arXiv2025

Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models

We introduce Sigma, an efficient large language model specialized for the system domain, empowered by a novel architecture including DiffQKV attention, and pre-trained on our meticulously collected system domain data. DiffQKV attention significantly enhances the inference efficiency of Sigma by optimizing the Query (Q), Key (K), and Value (V) components in the attention mechanism differentially, based on their varying impacts on the model performance and efficiency indicators. Specifically, we (1) conduct extensive experiments that demonstrate the model's varying sensitivity to the compression of K and V components, leading to the development of differentially compressed KV, and (2) propose augmented Q to expand the Q head dimension, which enhances the model's representation capacity with minimal impacts on the inference speed. Rigorous theoretical and empirical analyses reveal that DiffQKV attention significantly enhances efficiency, achieving up to a 33.36% improvement in inference speed over the conventional grouped-query attention (GQA) in long-context scenarios. We pre-train Sigma on 6T tokens from various sources, including 19.5B system domain data that we carefully collect and 1T tokens of synthesized and rewritten data. In general domains, Sigma achieves comparable performance to other state-of-arts models. In the system domain, we introduce the first comprehensive benchmark AIMicius, where Sigma demonstrates remarkable performance across all tasks, significantly outperforming GPT-4 with an absolute improvement up to 52.5%.

cs.CL↗

arXiv2025

MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data

Brain decoding aims to reconstruct visual perception of human subject from fMRI signals, which is crucial for understanding brain's perception mechanisms. Existing methods are confined to the single-subject paradigm due to substantial brain variability, which leads to weak generalization across individuals and incurs high training costs, exacerbated by limited availability of fMRI data. To address these challenges, we propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding from limited fMRI data. The proposed MindAligner enjoys several merits. First, we learn a Brain Transfer Matrix (BTM) that projects the brain signals of an arbitrary new subject to one of the known subjects, enabling seamless use of pre-trained decoding models. Second, to facilitate reliable BTM learning, a Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli with a multi-level brain alignment loss, uncovering fine-grained functional correspondences with high interpretability. Experiments indicate that MindAligner not only outperforms existing methods in visual decoding under data-limited conditions, but also provides valuable neuroscience insights in cross-subject functional analysis. The code will be made publicly available.

cs.CV↗

arXiv2025

Forward and backward problems for coupled subdiffusion systems

In this article, we investigate both forward and backward problems for coupled systems of time-fractional diffusion equations, encompassing scenarios of strong coupling. For the forward problem, we establish the well-posedness of the system, leveraging the eigensystem of the corresponding elliptic system as the foundation. When considering the backward problem, specifically the determination of initial values through final time observations, we demonstrate a Lipschitz stability estimate, which is consistent with the stability observed in the case of a single equation. To numerically address this backward problem, we refer to the explicit formulation of Tikhonov regularization to devise a multi-channel neural network architecture. This innovative architecture offers a versatile approach, exhibiting its efficacy in multidimensional settings through numerical examples and its robustness in handling initial values that have not been trained.

math.AP↗

arXiv2025

Two Channels of Metal-Rich Compact Stellar System Formation: Starbursts under High Ram Pressure versus Tidal Stripping

Most galaxies follow well-defined scaling relations of metallicity and stellar mass; however, some outliers at the low mass end of the observed galaxy population exhibit unusually high metallicity for their mass. Understanding how these objects get to be so metal-rich is vital for understanding the role of feedback in galaxy formation. Using the TNG50 simulation, we explore the origins of this phenomenon. We identify 227 metal-rich, compact stellar systems (CSSs) that deviate significantly from this scaling relation. These CSSs are satellites located in the vicinity of massive host galaxies, with stellar masses ranging from $10^{8} M_{\odot}$ to $10^{10}\ M_{\odot}$ (including six systems that are close analogs of the M31-M32 system). Contrary to the previously assumed scenario that such objects are predominantly products of tidal stripping, our results suggest a more prevalent role for ram pressure in their formation. Indeed, 76% (173) of these CSSs are formed through a burst of star formation occurring around the time of the first pericentric passage, typically at redshifts $z\lesssim1$, aided by strong ram pressure and tidal forces. The high ram pressure, resulting from the CSSs' rapid motion near the halo center, facilitates metal enrichment, producing high-metallicity CSSs by confining the metal-rich gas from bursty star formation, which leads to distinct stellar populations characterized by enhanced metallicity as well as high $α$-abundance. Only the remaining 24% (54) of metal-rich CSSs are generated through the tidal stripping of massive progenitors. Our results further indicate that M32 is more likely to have formed through intense star formation events rather than through gradual, tidal stripping, thereby providing crucial insights into the nature of low mass, compact galaxy formation.

astro-ph.GA↗

arXiv2025

Stable inversion of potential in nonlinear wave equations with cubic nonlinearity

This paper investigates inverse potential problems of wave equations with cubic nonlinearity. We develop a methodology for establishing stability estimates for inversion of lower order coefficients. The new ingredients of our approach include trilinear approximations of nonlinear response operators, symbol estimates of distorted plane waves, and lower order symbol calculus.

math.AP↗

arXiv2024

Function and derivative approximation by shallow neural networks

We investigate a Tikhonov regularization scheme specifically tailored for shallow neural networks within the context of solving a classic inverse problem: approximating an unknown function and its derivatives within a unit cubic domain based on noisy measurements. The proposed Tikhonov regularization scheme incorporates a penalty term that takes three distinct yet intricately related network (semi)norms: the extended Barron norm, the variation norm, and the Radon-BV seminorm. These choices of the penalty term are contingent upon the specific architecture of the neural network being utilized. We establish the connection between various network norms and particularly trace the dependence of the dimensionality index, aiming to deepen our understanding of how these norms interplay with each other. We revisit the universality of function approximation through various norms, establish rigorous error-bound analysis for the Tikhonov regularization scheme, and explicitly elucidate the dependency of the dimensionality index, providing a clearer understanding of how the dimensionality affects the approximation performance and how one designs a neural network with diverse approximating tasks.

math.NA↗

arXiv2024

Comparative Analysis of Control Observer-Based Methods for State Estimation of Lithium-Ion Batteries in Practical Scenarios

The reliability, lower computational complexity, and ease of implementation of control observers make them one of the most promising methods for the state estimation of Li-ion batteries (LIBs) in commercial applications. To pave their way, this study performs a comprehensive and systematic evaluation of four main categories of control observer-based methods in different practical scenarios considering estimation accuracy, computational time convergence speed, stability, and robustness against measurement uncertainties. Observers are designed using a second-order equivalent circuit model whose observability against different scenarios is rigorously investigated to verify the feasibility of the proposed analysis. Established techniques then are validated against driving datasets and their comparative usefulness is evaluated using an experimental setup. The analysis also evaluates the adaptability of different techniques to electric vehicle field data. The results indicate better accuracy, stability, robustness, and faster convergence for the PI and PID, while the estimations of the Luenberger observers find it hard to converge against highly dynamic loadfiles. Moreover, this study also discusses the sensitivity of observer-based techniques to battery ohmic polarization and voltage-related measurement uncertainties. The most remarkable contribution of the proposed study lies in providing guidance for researchers when choosing the control observers for online state estimation of LIBs.

eess.SY↗

arXiv2024

Semi-implicit Continuous Newton Method for Power Flow Analysis

As an effective emulator of ill-conditioned power flow, continuous Newton methods (CNMs) have been extensively investigated using explicit and implicit numerical integration algorithms. Explicit CNMs are prone to non-convergence issues due to their limited stable region, while implicit CNMs introduce additional iteration-loops of nonlinear equations. Faced with this, we propose a semi-implicit version of CNM. We formulate the power flow equations as a set of differential algebraic equations (DAEs), and solve the DAEs with the stiffly accurate Rosenbrock type method (SARM). The proposed method succeeds the numerical robustness from the implicit CNM framework while prevents the iterative solution of nonlinear systems, hence revealing higher convergence speed and computation efficiency. A new 4-stage 3rd-order hyper-stable SARM, together with a 2nd-order embedded formula to control the step size, is constructed to further accelerate convergence by tuning the damping factor. Case studies on ill-conditioned systems verified the alleged performance. An algorithm extension for MATPOWER is made available on Github for benchmarking.

eess.SY↗

arXiv2024

On the Solution Uniqueness of Data-Driven Modeling of Flexible Loads (with Supplementary Material)

This letter first explores the solution uniqueness of the data-driven modeling of price-responsive flexible loads (PFL). The PFL on the demand side is critical in modern power systems. An accurate PFL model is fundamental for system operations. However, whether the PFL model can be uniquely and correctly identified from operational data remains unclear. To address this, we analyze the structural and practical identifiability of the PFL model, deriving the dataset condition that guarantees the solution uniqueness. Besides, we point out the practical implications of the results. Numerical tests validate this work.

eess.SY↗

arXiv2024

Quantify Gas-to-Power Fault Propagation Speed:A Semi-Implicit Simulation Approach

Relying heavily on the secure supply of natural gas, the modern clean electric power systems are prone to the gas disturbances induced by the inherent rupture and leakage faults. For the first time, this paper studies the cross-system propagation speed of these faults using a simulation-based approach. Firstly, we establish the differential algebraic equation models of the rupture and leakage faults respectively. The boundary conditions at the fault locations are derived using the method of characteristics. Secondly, we propose utilizing a semi-implicit approach to perform post-fault simulations. The approach, based on the stiffly-accurate Rosenbrock scheme, possesses the implicit numerical stability and explicit computation burdens. Therefore, the high-dimensional and multi-time-scale stiff models can be solved in an efficient and robust way. Thirdly, to accurately locate the simulation events, which can not be predicted a priori, we propose a critical-time-location strategy based on the continuous Runge-Kutta approach. In case studies, we verified the accuracy and the efficiency superiority of the proposed simulation approach. The impacts of gas faults on gas and power dynamics were investigated by simulation, where the critical events were identified accurately. We found that the fault propagation speed mainly depends on the fault position and is influenced by the pipe frictions. The bi-directional coupling between gas and power may lead to cascading failures.

eess.SY↗

arXiv2024

Contextualized Data-Wrangling Code Generation in Computational Notebooks

Data wrangling, the process of preparing raw data for further analysis in computational notebooks, is a crucial yet time-consuming step in data science. Code generation has the potential to automate the data wrangling process to reduce analysts' overhead by translating user intents into executable code. Precisely generating data wrangling code necessitates a comprehensive consideration of the rich context present in notebooks, including textual context, code context and data context. However, notebooks often interleave multiple non-linear analysis tasks into linear sequence of code blocks, where the contextual dependencies are not clearly reflected. Directly training models with source code blocks fails to fully exploit the contexts for accurate wrangling code generation. To bridge the gap, we aim to construct a high quality datasets with clear and rich contexts to help training models for data wrangling code generation tasks. In this work, we first propose an automated approach, CoCoMine to mine data-wrangling code generation examples with clear multi-modal contextual dependency. It first adopts data flow analysis to identify the code blocks containing data wrangling codes. Then, CoCoMine extracts the contextualized datawrangling code examples through tracing and replaying notebooks. With CoCoMine, we construct CoCoNote, a dataset containing 58,221 examples for Contextualized Data-wrangling Code generation in Notebooks. To demonstrate the effectiveness of our dataset, we finetune a range of pretrained code models and prompt various large language models on our task. Furthermore, we also propose DataCoder, which encodes data context and code&textual contexts separately to enhance code generation. Experiment results demonstrate the significance of incorporating data context in data-wrangling code generation and the effectiveness of our model. We release code and data at url...

cs.SE↗

arXiv2024

Nonlinear PDE Constrained Optimal Dispatch of Gas and Power: A Global Linearization Approach

The coordinated dispatch of power and gas in the electricity-gas integrated energy system (EG-IES) is fundamental for ensuring operational security. However, the gas dynamics in the natural gas system (NGS) are governed by the nonlinear partial differential equations (PDE), making the dispatch problem of the EG-IES a complicated optimization model constrained by nonlinear PDE. To address it, we propose a globally linearized gas network model based on the Koopman operator theory, avoiding the commonly used local linearization and spatial discretization. Particularly, we propose a data-driven Koopman operator approximation approach for the globally linearized gas network model based on the extended dynamic mode decomposition, in which a physics-informed stability constraint is derived and embedded to improve the generalization ability and accuracy of the model. Based on this, we develop an optimal dispatch model for the EG-IES that first considers the nonlinear gas dynamics in the NGS. The case study verifies the effectiveness of this work. Simulation results reveal that the commonly used locally linearized gas network model fails to accurately capture the dynamic characteristics of NGS, bringing potential security threats to the system.

eess.SY↗

arXiv2024

High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures

Convolution is the core component within deep neural networks and it is computationally intensive and time consuming. Tensor data layouts significantly impact convolution operations in terms of memory access and computational efficiency. Yet, there is still a lack of comprehensive performance characterization on data layouts on SIMD architectures concerning convolution methods. This paper proposes three novel data layouts for im2win convolution: NHWC, CHWN, and CHWN8, and introduces a set of general optimization techniques for both direct and im2win convolutions. We compare the optimized im2win convolution with the direct convolution and PyTorch's im2col-based convolution across the aforementioned layouts on SIMD machines. The experiments demonstrated that the im2win convolution with the new NHWC layout achieved up to 355% performance speedup over NCHW layout. Our optimizations also significantly improve the performance of both im2win and direct convolutions. Our optimized im2win and direct convolutions achieved up to 95% and 94% of machine's theoretical peak performance, respectively.

cs.LG↗

arXiv2024

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

Large language models (LLMs) have exhibited remarkable ability in code generation. However, generating the correct solution in a single attempt still remains a challenge. Prior works utilize verification properties in software engineering to verify and re-rank solutions in a majority voting manner. But the assumption behind them that generated verification properties have better qualities than solutions may not always hold. In this paper, we treat them equally as different perspectives of LLMs' reasoning processes. We propose the Multi-Perspective Self-Consistency (MPSC) framework incorporating both inter- and intra-consistency across outputs from multiple perspectives. Specifically, we prompt LLMs to generate diverse outputs from three perspectives, Solution, Specification and Test case, constructing a 3-partite graph. With two measure functions of consistency, we embed both inter- and intra-consistency information into the graph. The optimal choice of solutions is then determined based on analysis in the graph. MPSC significantly boosts performance of foundation models (ChatGPT in this paper) on various benchmarks, including HumanEval (+15.91%), MBPP (+6.43%) and CodeContests (+9.37%), even surpassing GPT-4.

cs.CL↗

arXiv2024

Selene: Pioneering Automated Proof in Software Verification

Ensuring correctness is a pivotal aspect of software engineering. Among the various strategies available, software verification offers a definitive assurance of correctness. Nevertheless, writing verification proofs is resource-intensive and manpower-consuming, and there is a great need to automate this process. We introduce Selene in this paper, which is the first project-level automated proof benchmark constructed based on the real-world industrial-level operating system microkernel, seL4. Selene provides a comprehensive framework for end-to-end proof generation and a lightweight verification environment. Our experimental results with advanced large language models (LLMs), such as GPT-3.5-turbo and GPT-4, highlight the capabilities of LLMs in the domain of automated proof generation. Additionally, our further proposed augmentations indicate that the challenges presented by Selene can be mitigated in future research endeavors.

cs.SE↗

arXiv2024

Competition-Level Problems are Effective LLM Evaluators

Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about these abilities and the potential data contamination problem recently. This paper aims to evaluate the reasoning capacities of LLMs, specifically in solving recent competition-level programming problems in Codeforces, which are expert-crafted and unique, requiring deep understanding and robust reasoning skills. We first provide a comprehensive evaluation of GPT-4's peiceived zero-shot performance on this task, considering various aspects such as problems' release time, difficulties, and types of errors encountered. Surprisingly, the peiceived performance of GPT-4 has experienced a cliff like decline in problems after September 2021 consistently across all the difficulties and types of problems, which shows the potential data contamination, as well as the challenges for any existing LLM to solve unseen complex reasoning problems. We further explore various approaches such as fine-tuning, Chain-of-Thought prompting and problem description simplification, unfortunately none of them is able to consistently mitigate the challenges. Through our work, we emphasis the importance of this excellent data source for assessing the genuine reasoning capabilities of LLMs, and foster the development of LLMs with stronger reasoning abilities and better generalization in the future.

cs.CL↗