arXiv Science⌕ Search

arXiv subjects

Fei Teng

Publications and source records attributed to Fei Teng.

At least 109 records · Page 6Linked to original sources

Self-supervised Contrastive Learning for Implicit Collaborative Filtering

Contrastive learning-based recommendation algorithms have significantly advanced the field of self-supervised recommendation, particularly with BPR as a representative ranking prediction task that dominates implicit collaborative filtering. However, the presence of false-positive and false-negative examples in recommendation systems hampers accurate preference learning. In this study, we propose a simple self-supervised contrastive learning framework that leverages positive feature augmentation and negative label augmentation to improve the self-supervisory signal. Theoretical analysis demonstrates that our learning method is equivalent to maximizing the likelihood estimation with latent variables representing user interest centers. Additionally, we establish an efficient negative label augmentation technique that samples unlabeled examples with a probability linearly dependent on their relative ranking positions, enabling efficient augmentation in constant time complexity. Through validation on multiple datasets, we illustrate the significant improvements our method achieves over the widely used BPR optimization objective while maintaining comparable runtime.

cs.IR↗

Task-Aware Machine Unlearning and Its Application in Load Forecasting

Data privacy and security have become a non-negligible factor in load forecasting. Previous researches mainly focus on training stage enhancement. However, once the model is trained and deployed, it may need to `forget' (i.e., remove the impact of) part of training data if the these data are found to be malicious or as requested by the data owner. This paper introduces the concept of machine unlearning which is specifically designed to remove the influence of part of the dataset on an already trained forecaster. However, direct unlearning inevitably degrades the model generalization ability. To balance between unlearning completeness and model performance, a performance-aware algorithm is proposed by evaluating the sensitivity of local model parameter change using influence function and sample re-weighting. Furthermore, we observe that the statistical criterion such as mean squared error, cannot fully reflect the operation cost of the downstream tasks in power system. Therefore, a task-aware machine unlearning is proposed whose objective is a trilevel optimization with dispatch and redispatch problems considered. We theoretically prove the existence of the gradient of such an objective, which is key to re-weighting the remaining samples. We tested the unlearning algorithms on linear, CNN, and MLP-Mixer based load forecasters with a realistic load dataset. The simulation demonstrates the balance between unlearning completeness and operational cost. All codes can be found at https://github.com/xuwkk/task_aware_machine_unlearning.

cs.LG↗

Out-of-Distribution Generalized Dynamic Graph Neural Network for Human Albumin Prediction

Human albumin is essential for indicating the body's overall health. Accurately predicting plasma albumin levels and determining appropriate doses are urgent clinical challenges, particularly in critically ill patients, to maintain optimal blood levels. However, human albumin prediction is non-trivial that has to leverage the dynamics of biochemical markers as well as the experience of treating patients. Moreover, the problem of distribution shift is often encountered in real clinical data, which may lead to a decline in the model prediction performance and reduce the reliability of the model's application. In this paper, we propose a framework named Out-of-Distribution Generalized Dynamic Graph Neural Network for Human Albumin Prediction (DyG-HAP), which is able to provide accurate albumin predictions for Intensity Care Unit (ICU) patients during hospitalization. We first model human albumin prediction as a dynamic graph regression problem to model the dynamics and patient relationship. Then, we propose a disentangled dynamic graph attention mechanism to capture and disentangle the patterns whose relationship to labels under distribution shifts is invariant and variant respectively. Last, we propose an invariant dynamic graph regression method to encourage the model to rely on invariant patterns to make predictions. Moreover, we propose a dataset named Albumin level testing and nutritional dosing data for Intensive Care (ANIC) for evaluation. Extensive experiments demonstrate the superiority of our method compared to several baseline methods in human albumin prediction.

cs.LG↗

Enhancing Cyber-Resiliency of DER-based SmartGrid: A Survey

The rapid development of information and communications technology has enabled the use of digital-controlled and software-driven distributed energy resources (DERs) to improve the flexibility and efficiency of power supply, and support grid operations. However, this evolution also exposes geographically-dispersed DERs to cyber threats, including hardware and software vulnerabilities, communication issues, and personnel errors, etc. Therefore, enhancing the cyber-resiliency of DER-based smart grid - the ability to survive successful cyber intrusions - is becoming increasingly vital and has garnered significant attention from both industry and academia. In this survey, we aim to provide a systematical and comprehensive review regarding the cyber-resiliency enhancement (CRE) of DER-based smart grid. Firstly, an integrated threat modeling method is tailored for the hierarchical DER-based smart grid with special emphasis on vulnerability identification and impact analysis. Then, the defense-in-depth strategies encompassing prevention, detection, mitigation, and recovery are comprehensively surveyed, systematically classified, and rigorously compared. A CRE framework is subsequently proposed to incorporate the five key resiliency enablers. Finally, challenges and future directions are discussed in details. The overall aim of this survey is to demonstrate the development trend of CRE methods and motivate further efforts to improve the cyber-resiliency of DER-based smart grid.

eess.SY↗

Quantum Field Theory, Worldline Theory, and Spin Magnitude Change in Orbital Evolution

A previous paper~\cite{Bern:2022kto} identified a puzzle stemming from the amplitudes-based approach to spinning bodies in general relativity: additional Wilson coefficients appear compared to current worldline approaches to conservative dynamics of generic astrophysical objects, including neutron stars. In this paper we clarify the nature of analogous Wilson coefficients in the simpler theory of electrodynamics. We analyze the original field-theory construction, identifying definite-spin states some of which have negative norms, and relating the additional Wilson coefficients in the classical theory to transitions between different quantum spin states. We produce a new version of the theory which also has additional Wilson coefficients, but no negative-norm states. We match, through $\mathcal O(α^2)$ and $\mathcal O(S^2)$, the Compton amplitudes of these field theories with those of a modified worldline theory with extra degrees of freedom introduced by releasing the spin supplementary condition. We build an effective two-body Hamiltonian that matches the impulse and spin kick of the modified field theory and of the worldline theory, displaying additional Wilson coefficients compared to standard worldline approaches. The results are then compactly expressed in terms of an eikonal formula. Our key conclusion is that, contrary to standard approaches, while the magnitude of the spin tensor is still conserved, the magnitude of the spin vector can change under conserved Hamiltonian dynamics and this change is governed by the additional Wilson coefficients. For specific values of Wilson coefficients the results are equivalent to those from a definite spin obeying the spin supplementary condition, but for generic values they are physically inequivalent. These results warrant detailed studies of the corresponding issues in general relativity.

hep-th↗

ULMA: Unified Language Model Alignment with Human Demonstration and Point-wise Preference

Aligning language models to human expectations, e.g., being helpful and harmless, has become a pressing challenge for large language models. A typical alignment procedure consists of supervised fine-tuning and preference learning. Most preference learning methods, such as RLHF and DPO, depend on pairwise preference data, which inadequately address scenarios where human feedback is point-wise, leading to potential information loss and suboptimal performance. Addressing this gap, we introduce Point-wise Direct Preference Optimization, a novel preference learning method designed to harness point-wise feedback effectively. Our work also uncovers a novel connection between supervised fine-tuning and point-wise preference learning, culminating in Unified Language Model Alignment, a single-step method that unifies the alignment with human demonstrations and point-wise preferences. Extensive experiments on point-wise preference datasets with binary or continuous labels validate the effectiveness of our methods. Our code and a new dataset with high-quality demonstration samples on harmlessness are released.

cs.LG↗

Scheduling of Software-Defined Microgrids for Optimal Frequency Regulation

Integrated with a high share of Inverter-Based Resources (IBRs), microgrids face increasing complexity of frequency dynamics, especially after unintentional islanding from the maingrid. These IBRs, on the other hand, provide more control flexibility to shape the frequency dynamics of microgrid and together with advanced communication infrastructure offer new opportunities in the future software-defined microgrids. To enhance the frequency stability of microgrids with high IBR penetration, this paper proposes an optimal scheduling framework for software-defined microgrids to maintain frequency stability by utilizing the non-essential load shedding and dynamical optimization of the virtual inertia and virtual damping from IBRs. Moreover, side effects of these services, namely, the time delay associated with non-essential load shedding and potential IBR control parameter update failure are explicitly modeled to avoid underestimations of frequency deviation and over-optimistic results. The effectiveness and significant economic value of the proposed simultaneous and dynamic virtual inertia and damping provision strategy are demonstrated based on case studies in the modified IEEE 33-bus system.

eess.SY↗

Pricing of Short Circuit Current in High IBR-Penetrated System

With the growing penetration of Inverter-Based Resources (IBRs) in power systems, stability service markets have emerged to incentivize technologies that ensure power system stability and reliability. Among the various challenges faced in power system operation and stability, a prominent issue raised from the increasing integration of large-scale IBRs is the significant reduction of the Short-Circuit Current (SCC) level in the system, which poses a considerable threat to system voltage stability and protection. Thus, a proper market mechanism to incentivize the provision of SCC as a stability service is desired. However, the pricing of this service within the future stability market has not yet been fully developed, due to the nonconvex nature of SCC constraints and the locational property of SCC. To address these problems, this work aims to explore, for the first time, a pricing model for SCC service by incorporating a linearized SCC constraint into the Unit Commitment (UC) problem, to achieve the desired SCC level and extract the shadow price for SCC through different pricing methods.

eess.SY↗

Learning Time-aware Graph Structures for Spatially Correlated Time Series Forecasting

Spatio-temporal forecasting of future values of spatially correlated time series is important across many cyber-physical systems (CPS). Recent studies offer evidence that the use of graph neural networks to capture latent correlations between time series holds a potential for enhanced forecasting. However, most existing methods rely on pre-defined or self-learning graphs, which are either static or unintentionally dynamic, and thus cannot model the time-varying correlations that exhibit trends and periodicities caused by the regularity of the underlying processes in CPS. To tackle such limitation, we propose Time-aware Graph Structure Learning (TagSL), which extracts time-aware correlations among time series by measuring the interaction of node and time representations in high-dimensional spaces. Notably, we introduce time discrepancy learning that utilizes contrastive learning with distance-based regularization terms to constrain learned spatial correlations to a trend sequence. Additionally, we propose a periodic discriminant function to enable the capture of periodic changes from the state of nodes. Next, we present a Graph Convolution-based Gated Recurrent Unit (GCGRU) that jointly captures spatial and temporal dependencies while learning time-aware and node-specific patterns. Finally, we introduce a unified framework named Time-aware Graph Convolutional Recurrent Network (TGCRN), combining TagSL, and GCGRU in an encoder-decoder architecture for multi-step spatio-temporal forecasting. We report on experiments with TGCRN and popular existing approaches on five real-world datasets, thus providing evidence that TGCRN is capable of advancing the state-of-the-art. We also cover a detailed ablation study and visualization analysis, offering detailed insight into the effectiveness of time-aware structure learning.

cs.LG↗

E2E-AT: A Unified Framework for Tackling Uncertainty in Task-aware End-to-end Learning

Successful machine learning involves a complete pipeline of data, model, and downstream applications. Instead of treating them separately, there has been a prominent increase of attention within the constrained optimization (CO) and machine learning (ML) communities towards combining prediction and optimization models. The so-called end-to-end (E2E) learning captures the task-based objective for which they will be used for decision making. Although a large variety of E2E algorithms have been presented, it has not been fully investigated how to systematically address uncertainties involved in such models. Most of the existing work considers the uncertainties of ML in the input space and improves robustness through adversarial training. We extend this idea to E2E learning and prove that there is a robustness certification procedure by solving augmented integer programming. Furthermore, we show that neglecting the uncertainty of COs during training causes a new trigger for generalization errors. To include all these components, we propose a unified framework that covers the uncertainties emerging in both the input feature space of the ML models and the COs. The framework is described as a robust optimization problem and is practically solved via end-to-end adversarial training (E2E-AT). Finally, the performance of E2E-AT is evaluated by a real-world end-to-end power system operation problem, including load forecasting and sequential scheduling tasks.

cs.LG↗

The Sub-Leading Scattering Waveform from Amplitudes

We compute the next-to-leading order term in the scattering waveform of uncharged black holes in classical general relativity and of half-BPS black holes in $\mathcal{N}=8$ supergravity. We propose criteria, generalizing explicit calculations at next-to-leading order, for determining the terms in amplitudes that contribute to local observables. For general relativity, we construct the relevant classical integrand through generalized unitarity in two distinct ways, (1) in a heavy-particle effective theory and (2) in general relativity minimally-coupled to scalar fields. With a suitable prescription for the matter propagator in the former, we find agreement between the two methods, thus demonstrating the absence of interference of quantum and classically-singular contributions. The classical $\mathcal{N}=8$ integrand for massive scalar fields is constructed through dimensional reduction of the known five-point one-loop integrand. Our calculation exhibits novel features compared to conservative calculations and inclusive observables, such as the appearance of master integrals with intersecting matter lines and the appearance of a classical infrared divergence whose absence from classical observables requires a suitable definition of the retarded time.

hep-th↗

Preventive-Corrective Cyber-Defense: Attack-Induced Region Minimization and Cybersecurity Margin Maximization

False data injection (FDI) cyber-attacks on power systems can be prevented by strategically selecting and protecting a sufficiently large measurement subset, which, however, requires adequate cyber-defense resources for measurement protection. With any given cyber-defense resource, this paper proposes a preventive-corrective cyber-defense strategy, which minimizes the FDI attack-induced region in a preventive manner, followed by maximizing the cybersecurity margin in a corrective manner. First, this paper proposes a preventive cyber-defense strategy that minimizes the volume of the FDI attack-induced region via preventive allocation of any given measurement protection resource. Particularly, a sufficient condition for constructing the FDI unattackable lines is proposed, indicating that the FDI cyber-attack could be locally rather than globally prevented. Then, given a non-empty FDI attack-induced region, this paper proposes a corrective cyber-defense strategy that maximizes the cybersecurity margin, leading to a trade-off between the safest-but-expensive operation point (i.e., Euclidean Chebyshev center) and the cheapest-but-dangerous operation point. Simulation results on a modified IEEE 14 bus system verify the effectiveness and cost-effectiveness of the proposed preventive-corrective cyber-defense strategy.

eess.SY↗

Cyber Recovery from Dynamic Load Altering Attacks: Linking Electricity, Transportation, and Cyber Networks

To address the increasing vulnerability of power grids, significant attention has been focused on the attack detection and impact mitigation. However, it is still unclear how to effectively and quickly recover the cyber and physical networks from a cyberattack. In this context, this paper presents the first investigation of the Cyber Recovery from Dynamic load altering Attack (CRDA). Considering the interconnection among electricity, transportation, and cyber networks, two essential sub-tasks are formulated for the CRDA: i) Optimal design of repair crew routes to remove installed malware and ii) Adaptive adjustment of system operation to eliminate the mitigation costs while guaranteeing stability. To achieve this, linear stability constraints are obtained by estimating the related eigenvalues under the variation of multiple IBR droop gains based on the sensitivity information of strategically selected sampling points. Moreover, to obtain the robust recovery strategy, the potential counter-measures from the adversary during the recovery process are modeled as maximizing the attack impact of remaining compromised resources in each step. A Mixed-Integer Linear Programming (MILP) problem can be finally formulated for the CRDA with the primary objective to reset involved droop gains and secondarily to repair all compromised loads. Case studies are performed in the modified IEEE 39-bus power system to illustrate the effectiveness of the proposed CRDA compared to the benchmark case.

eess.SY↗

Optimal Design of Neural Network Structure for Power System Frequency Security Constraints

Recently, frequency security is challenged by high uncertainty and low inertia in power system with high penetration of Renewable Energy Sources (RES). In the context of Unit Commitment (UC) problems, frequency security constraints represented by neural networks have been developed and embedded into the optimization problem to represent complicated frequency dynamics. However, there are two major disadvantages related to this technique: the risk of overconfident prediction and poor computational efficiency. To handle these disadvantages, novel methodologies are proposed to optimally design the neural network structure, including the use of asymmetric loss function during the training stage and scientifically selecting neural network size and topology. The effectiveness of the proposed methodologies are validated by case study which reveals the improvement of conservativeness and mitigation of computation performance issues.

eess.SY↗

Vehicle-to-grid plug-in forecasting for participation in ancillary services markets

Electric vehicle (EV) charge points (CPs) can be used by aggregators to provide frequency response (FR) services. Aggregators must have day-ahead half-hourly forecasts of minimum aggregate vehicle-to-grid (V2G) plug-in to produce meaningful bids for the day-ahead ancillary services market. However, there is a lack of understanding on what features should be considered and how complex the forecasting model should be. This paper explores the dependency of aggregate V2G plug-in on historic plug-in levels, calendar variables, and weather conditions. These investigations are used to develop three day-ahead forecasts of minimum aggregate V2G plug-in during 30-minute window. A neural network that considers previous V2G plug-in values the day before, three days before, and seven days before, in addition to day of the week, month, and hour, is found to be the most accurate.

eess.SY↗

A Missing Value Filling Model Based on Feature Fusion Enhanced Autoencoder

With the advent of the big data era, the data quality problem is becoming more critical. Among many factors, data with missing values is one primary issue, and thus developing effective imputation models is a key topic in the research community. Recently, a major research direction is to employ neural network models such as self-organizing mappings or automatic encoders for filling missing values. However, these classical methods can hardly discover interrelated features and common features simultaneously among data attributes. Especially, it is a very typical problem for classical autoencoders that they often learn invalid constant mappings, which dramatically hurts the filling performance. To solve the above-mentioned problems, we propose a missing-value-filling model based on a feature-fusion-enhanced autoencoder. We first incorporate into an autoencoder a hidden layer that consists of de-tracking neurons and radial basis function neurons, which can enhance the ability of learning interrelated features and common features. Besides, we develop a missing value filling strategy based on dynamic clustering that is incorporated into an iterative optimization process. This design can enhance the multi-dimensional feature fusion ability and thus improves the dynamic collaborative missing-value-filling performance. The effectiveness of the proposed model is validated by extensive experiments compared to a variety of baseline methods on thirteen data sets.

cs.LG↗

Control-mode as a Grid Service in Software-defined Power Grids: GFL vs GFM

In power systems with high penetration of power electronics, grid-forming control is proposed to replace traditional Grid-Following Converter (GFL) in order to improve the overall system strength and resist small-signal instability in weak grids by directly forming the terminal voltage. However, sufficient headroom of both active and reactive power must be made available for Grid-Forming Converter (GFM) to operate, potentially leading to sub-optimal operation in steady states. This presents a new research problem to optimally allocate between GFM and GFL to balance the ability of GFMs to improve the grid strength and the potential economic loss resulting from reserved headroom. An optimization framework under software-defined grids is proposed, for the first time, to dynamically determine the optimal allocation of GFMs and GFLs in power systems at each time step of system scheduling according to system conditions, which ensures both system stability and minimum operational cost. To achieve this, the system scheduling model is expanded to simultaneously consider the constraints related to active and reactive power reserves for GFMs, as well as the system level stability. Case studies conducted on the modified IEEE 30-bus system demonstrate significant economic benefits in that the optimal proportion of GFMs in the power system can be dynamically determined while ensuring power reserve and grid stability constraints.

eess.SY↗

Benchmarking Explanatory Models for Inertia Forecasting using Public Data of the Nordic Area

This paper investigates the performance of a day-ahead explanatory model for inertia forecasting based on field data in the Nordic system, which achieves a 43% reduction in mean absolute percentage error (MAPE) against a state-of-the-art time-series forecast model. The generalizability of the explanatory model is verified by its consistent performance on Nordic and Great Britain datasets. Also, it appears that a long duration of training data is not required to obtain accurate results with this model, but taking a more spatially granular approach reduces the MAPE by 3.6%. Finally, two further model enhancements are studied considering the specific features in Nordic system: (i) a monthly interaction variable applied to the day-ahead national demand forecast feature, reducing the MAPE by up to 18%; and (ii) a feature based on the inertia from hydropower, although this has a negligible impact. The field dataset used for benchmarking is also made publicly available.

eess.SY↗