arXiv subjects

Fei Wang

Publications and source records attributed to Fei Wang.

At least 397 records · Page 22Linked to original sources

arXiv2024

Ensemble Variational Quantum Algorithm for Non-Markovian Quantum Dynamics

Many physical and chemical processes in the condensed phase environment exhibit non-Markovian quantum dynamics. As such simulations are challenging on classical computers, we developed a variational quantum algorithm that is capable of simulating non-Markovian dynamics on NISQ devices. We used a quantum system linearly coupled to its harmonic bath as the model Hamiltonian. The non-Markovianity is captured by introducing auxiliary variables from the bath trajectories. With Monte Carlo sampling of the bath degrees of freedom, finite temperature dynamics is produced. We validated the algorithm on the simulator and demonstrated its performance on the IBM quantum device. The framework developed is naturally adapted to any anharmonic bath with non-linear coupling to the system, and is also well suited for simulating spin chain dynamics in a dissipative environment.

quant-ph↗

arXiv2024

Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking

While large language models (LLMs) have demonstrated increasing power, they have also given rise to a wide range of harmful behaviors. As representatives, jailbreak attacks can provoke harmful or unethical responses from LLMs, even after safety alignment. In this paper, we investigate a novel category of jailbreak attacks specifically designed to target the cognitive structure and processes of LLMs. Specifically, we analyze the safety vulnerability of LLMs in the face of (1) multilingual cognitive overload, (2) veiled expression, and (3) effect-to-cause reasoning. Different from previous jailbreak attacks, our proposed cognitive overload is a black-box attack with no need for knowledge of model architecture or access to model weights. Experiments conducted on AdvBench and MasterKey reveal that various LLMs, including both popular open-source model Llama 2 and the proprietary model ChatGPT, can be compromised through cognitive overload. Motivated by cognitive psychology work on managing cognitive load, we further investigate defending cognitive overload attack from two perspectives. Empirical studies show that our cognitive overload from three perspectives can jailbreak all studied LLMs successfully, while existing defense strategies can hardly mitigate the caused malicious uses effectively.

cs.CL↗

arXiv2024

In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking

Deep learning-based methods monopolize the latest research in the field of thermal infrared (TIR) object tracking. However, relying solely on deep learning models to obtain better tracking results requires carefully selecting feature information that is beneficial to representing the target object and designing a reasonable template update strategy, which undoubtedly increases the difficulty of model design. Thus, recent TIR tracking methods face many challenges in complex scenarios. This paper introduces a novel Deep Bayesian Filtering (DBF) method to enhance TIR tracking in these challenging situations. DBF is distinctive in its dual-model structure: the system and observation models. The system model leverages motion data to estimate the potential positions of the target object based on two-dimensional Brownian motion, thus generating a prior probability. Following this, the observation model comes into play upon capturing the TIR image. It serves as a classifier and employs infrared information to ascertain the likelihood of these estimated positions, creating a likelihood probability. According to the guidance of the two models, the position of the target object can be determined, and the template can be dynamically updated. Experimental analysis across several benchmark datasets reveals that DBF achieves competitive performance, surpassing most existing TIR tracking methods in complex scenarios.

cs.CV↗

arXiv2024

Tree-Guided Rare Feature Selection and Logic Aggregation with Electronic Health Records Data

Statistical learning with a large number of rare binary features is commonly encountered in analyzing electronic health records (EHR) data, especially in the modeling of disease onset with prior medical diagnoses and procedures. Dealing with the resulting highly sparse and large-scale binary feature matrix is notoriously challenging as conventional methods may suffer from a lack of power in testing and inconsistency in model fitting while machine learning methods may suffer from the inability of producing interpretable results or clinically-meaningful risk factors. To improve EHR-based modeling and utilize the natural hierarchical structure of disease classification, we propose a tree-guided feature selection and logic aggregation approach for large-scale regression with rare binary features, in which dimension reduction is achieved through not only a sparsity pursuit but also an aggregation promoter with the logic operator of ``or''. We convert the combinatorial problem into a convex linearly-constrained regularized estimation, which enables scalable computation with theoretical guarantees. In a suicide risk study with EHR data, our approach is able to select and aggregate prior mental health diagnoses as guided by the diagnosis hierarchy of the International Classification of Diseases. By balancing the rarity and specificity of the EHR diagnosis records, our strategy improves both prediction and model interpretation. We identify important higher-level categories and subcategories of mental health conditions and simultaneously determine the level of specificity needed for each of them in predicting suicide risk.

cs.LG↗

arXiv2024

Medical Knowledge Graph QA for Drug-Drug Interaction Prediction based on Multi-hop Machine Reading Comprehension

Drug-drug interaction prediction is a crucial issue in molecular biology. Traditional methods of observing drug-drug interactions through medical experiments require significant resources and labor. This paper presents a medical knowledge graph question answering model, dubbed MedKGQA, that predicts drug-drug interaction by employing machine reading comprehension from closed-domain literature and constructing a knowledge graph of drug-protein triplets from open-domain documents. The model vectorizes the drug-protein target attributes in the graph using entity embeddings and establishes directed connections between drug and protein entities based on the metabolic interaction pathways of protein targets in the human body. This aligns multiple external knowledge and applies it to learn the graph neural network. Without bells and whistles, the proposed model achieved a 4.5% improvement in terms of drug-drug interaction prediction accuracy compared to previous state-of-the-art models on the Qangaroo MedHop dataset. Experimental results demonstrate the efficiency and effectiveness of the model and verify the feasibility of integrating external knowledge in machine reading comprehension tasks.

cs.CL↗

arXiv2024

MapTrack: Tracking in the Map

Multi-Object Tracking (MOT) aims to maintain stable and uninterrupted trajectories for each target. Most state-of-the-art approaches first detect objects in each frame and then implement data association between new detections and existing tracks using motion models and appearance similarities. Despite achieving satisfactory results, occlusion and crowds can easily lead to missing and distorted detections, followed by missing and false associations. In this paper, we first revisit the classic tracker DeepSORT, enhancing its robustness over crowds and occlusion significantly by placing greater trust in predictions when detections are unavailable or of low quality in crowded and occluded scenes. Specifically, we propose a new framework comprising of three lightweight and plug-and-play algorithms: the probability map, the prediction map, and the covariance adaptive Kalman filter. The probability map identifies whether undetected objects have genuinely disappeared from view (e.g., out of the image or entered a building) or are only temporarily undetected due to occlusion or other reasons. Trajectories of undetected targets that are still within the probability map are extended by state estimations directly. The prediction map determines whether an object is in a crowd, and we prioritize state estimations over observations when severe deformation of observations occurs, accomplished through the covariance adaptive Kalman filter. The proposed method, named MapTrack, achieves state-of-the-art results on popular multi-object tracking benchmarks such as MOT17 and MOT20. Despite its superior performance, our method remains simple, online, and real-time. The code will be open-sourced later.

cs.CV↗

arXiv2024

Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm

Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow the proficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novel response-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness.

cs.AI↗

arXiv2024

Rethinking Large Language Model Architectures for Sequential Recommendations

Recently, sequential recommendation has been adapted to the LLM paradigm to enjoy the power of LLMs. LLM-based methods usually formulate recommendation information into natural language and the model is trained to predict the next item in an auto-regressive manner. Despite their notable success, the substantial computational overhead of inference poses a significant obstacle to their real-world applicability. In this work, we endeavor to streamline existing LLM-based recommendation models and propose a simple yet highly effective model Lite-LLM4Rec. The primary goal of Lite-LLM4Rec is to achieve efficient inference for the sequential recommendation task. Lite-LLM4Rec circumvents the beam search decoding by using a straight item projection head for ranking scores generation. This design stems from our empirical observation that beam search decoding is ultimately unnecessary for sequential recommendations. Additionally, Lite-LLM4Rec introduces a hierarchical LLM structure tailored to efficiently handle the extensive contextual information associated with items, thereby reducing computational overhead while enjoying the capabilities of LLMs. Experiments on three publicly available datasets corroborate the effectiveness of Lite-LLM4Rec in both performance and inference efficiency (notably 46.8% performance improvement and 97.28% efficiency improvement on ML-1m) over existing LLM-based methods. Our implementations will be open sourced.

cs.IR↗

arXiv2024

Privacy-Preserving Language Model Inference with Instance Obfuscation

Language Models as a Service (LMaaS) offers convenient access for developers and researchers to perform inference using pre-trained language models. Nonetheless, the input data and the inference results containing private information are exposed as plaintext during the service call, leading to privacy issues. Recent studies have started tackling the privacy issue by transforming input data into privacy-preserving representation from the user-end with the techniques such as noise addition and content perturbation, while the exploration of inference result protection, namely decision privacy, is still a blank page. In order to maintain the black-box manner of LMaaS, conducting data privacy protection, especially for the decision, is a challenging task because the process has to be seamless to the models and accompanied by limited communication and computation overhead. We thus propose Instance-Obfuscated Inference (IOI) method, which focuses on addressing the decision privacy issue of natural language understanding tasks in their complete life-cycle. Besides, we conduct comprehensive experiments to evaluate the performance as well as the privacy-protection strength of the proposed method on various benchmarking tasks.

cs.CL↗

arXiv2024

Breakup of liquid jets: Thermodynamic perspectives

Breakup of a liquid jet into a chain of droplets is common in nature and industry. Previous researchers developed profound mathematic and fluid dynamic models to address this breakup phenomenon starting from tiny perturbations. However, the morphological evolution of the jets at large amplitude perturbations is still an open question. Here, we report a concise thermodynamic model based on the surface area minimization principle. Our results demonstrate a reversible breakup transition from a continuous jet via droplets towards a uniform-radius cylinder, which has not been found previously, but is observed in our simulations. This new observation is attributed to a geometric constraint, which was often overlooked in former studies. We anticipate our model to be a shortcut to tackle many similar highly nonlinear morphological evolutions without solving abstruse fluid dynamic equations for inviscid fluids.

physics.flu-dyn↗

arXiv2024

Efficient initialization of fluxonium qubits based on auxiliary energy levels

Fast and high-fidelity qubit initialization is crucial for low-frequency qubits such as fluxonium, and in applications of many quantum algorithms and quantum error correction codes. In a circuit quantum electrodynamics system, the initialization is typically achieved by transferring the state between the qubit and a short-lived cavity through microwave driving, also known as the sideband cooling process in atomic system. Constrained by the selection rules from the parity symmetry of the wavefunctions, the sideband transitions are only enabled by multi-photon processes which requires multi-tone or strong driving. Leveraging the flux-tunability of fluxonium, we circumvent this limitation by breaking flux symmetry to enable an interaction between a non-computational qubit transition and the cavity excitation. With single-tone sideband driving, we realize qubit initialization with a fidelity exceeding 99% within a duration of 300 ns, robust against the variation of control parameters. Furthermore, we show that our initialization scheme has a built-in benefit in simultaneously removing the second-excited state population of the qubit, and can be easily incorporated into a large-scale fluxonium processor.

quant-ph↗

arXiv2024

A new class of composite GBII regression models with varying threshold for modelling heavy-tailed data

The four-parameter generalized beta distribution of the second kind (GBII) has been proposed for modelling insurance losses with heavy-tailed features. The aim of this paper is to present a parametric composite GBII regression modelling by splicing two GBII distributions using mode matching method. It is designed for simultaneous modeling of small and large claims and capturing the policyholder heterogeneity by introducing the covariates into the location parameter. In such cases, the threshold that splits two GBII distributions varies across individuals policyholders based on their risk features. The proposed regression modelling also contains a wide range of insurance loss distributions as the head and the tail respectively and provides the close-formed expressions for parameter estimation and model prediction. A simulation study is conducted to show the accuracy of the proposed estimation method and the flexibility of the regressions. Some illustrations of the applicability of the new class of distributions and regressions are provided with a Danish fire losses data set and a Chinese medical insurance claims data set, comparing with the results of competing models from the literature.

stat.ME↗

arXiv2024

Improving Fairness of Automated Chest X-ray Diagnosis by Contrastive Learning

Purpose: Limited studies exploring concrete methods or approaches to tackle and enhance model fairness in the radiology domain. Our proposed AI model utilizes supervised contrastive learning to minimize bias in CXR diagnosis. Materials and Methods: In this retrospective study, we evaluated our proposed method on two datasets: the Medical Imaging and Data Resource Center (MIDRC) dataset with 77,887 CXR images from 27,796 patients collected as of April 20, 2023 for COVID-19 diagnosis, and the NIH Chest X-ray (NIH-CXR) dataset with 112,120 CXR images from 30,805 patients collected between 1992 and 2015. In the NIH-CXR dataset, thoracic abnormalities include atelectasis, cardiomegaly, effusion, infiltration, mass, nodule, pneumonia, pneumothorax, consolidation, edema, emphysema, fibrosis, pleural thickening, or hernia. Our proposed method utilizes supervised contrastive learning with carefully selected positive and negative samples to generate fair image embeddings, which are fine-tuned for subsequent tasks to reduce bias in chest X-ray (CXR) diagnosis. We evaluated the methods using the marginal AUC difference ($δ$ mAUC). Results: The proposed model showed a significant decrease in bias across all subgroups when compared to the baseline models, as evidenced by a paired T-test (p<0.0001). The $δ$ mAUC obtained by our method were 0.0116 (95\% CI, 0.0110-0.0123), 0.2102 (95% CI, 0.2087-0.2118), and 0.1000 (95\% CI, 0.0988-0.1011) for sex, race, and age on MIDRC, and 0.0090 (95\% CI, 0.0082-0.0097) for sex and 0.0512 (95% CI, 0.0512-0.0532) for age on NIH-CXR, respectively. Conclusion: Employing supervised contrastive learning can mitigate bias in CXR diagnosis, addressing concerns of fairness and reliability in deep learning-based diagnostic methods.

eess.IV↗

arXiv2023

Rethinking Tabular Data Understanding with Large Language Models

Large Language Models (LLMs) have shown to be capable of various tasks, yet their capability in interpreting and reasoning over tabular data remains an underexplored area. In this context, this study investigates from three core perspectives: the robustness of LLMs to structural perturbations in tables, the comparative analysis of textual and symbolic reasoning on tables, and the potential of boosting model performance through the aggregation of multiple reasoning pathways. We discover that structural variance of tables presenting the same content reveals a notable performance decline, particularly in symbolic reasoning tasks. This prompts the proposal of a method for table structure normalization. Moreover, textual reasoning slightly edges out symbolic reasoning, and a detailed error analysis reveals that each exhibits different strengths depending on the specific tasks. Notably, the aggregation of textual and symbolic reasoning pathways, bolstered by a mix self-consistency mechanism, resulted in achieving SOTA performance, with an accuracy of 73.6% on WIKITABLEQUESTIONS, representing a substantial advancement over previous existing table processing paradigms of LLMs.

cs.CL↗

arXiv2023

Flipped SU(6) Unification of the $SU(3)_c\times SU(3)_L\times U(1)_X$ Model

We propose to partially unify the sequential $SU(3)_c\times SU(3)_L\times U(1)_X$ model (with $β=1/\sqrt{3}$) into the flipped $SU(6)$ model with the gauge group $SU(6)\times U(1)_K$. Gauge anomaly cancellation can easily be satisfied. We discuss the relevant Higgs sector, the low energy $331$ model spectrum and the unification of $SU(3)_c$ and $SU(3)_L$ gauge couplings. Neutrino masses generation and successful gauge coupling unification can set lower/upper bounds on the $331$ breaking scale. The partial proton decay lifetime of various channels, for example, the $p\to e^+ π^0$ channel, in flipped SU(6) GUT are discussed. We find that certain parameter region with $M_{331}\sim 10^{15}$ GeV of case II (for case with $M_{331}$ scale ${\bf \tilde{H}_{3,8}}$ Higgs field) can predict a partial proton lifetime of order $10^{34}$ years for $p\to e^+ π^0$ mode, which can be tested soon by future DUNE and Hyper-Kamiokande experiments.

hep-ph↗

arXiv2023

AdapTraj: A Multi-Source Domain Generalization Framework for Multi-Agent Trajectory Prediction

Multi-agent trajectory prediction, as a critical task in modeling complex interactions of objects in dynamic systems, has attracted significant research attention in recent years. Despite the promising advances, existing studies all follow the assumption that data distribution observed during model learning matches that encountered in real-world deployments. However, this assumption often does not hold in practice, as inherent distribution shifts might exist in the mobility patterns for deployment environments, thus leading to poor domain generalization and performance degradation. Consequently, it is appealing to leverage trajectories from multiple source domains to mitigate such discrepancies for multi-agent trajectory prediction task. However, the development of multi-source domain generalization in this task presents two notable issues: (1) negative transfer; (2) inadequate modeling for external factors. To address these issues, we propose a new causal formulation to explicitly model four types of features: domain-invariant and domain-specific features for both the focal agent and neighboring agents. Building upon the new formulation, we propose AdapTraj, a multi-source domain generalization framework specifically tailored for multi-agent trajectory prediction. AdapTraj serves as a plug-and-play module that is adaptable to a variety of models. Extensive experiments on four datasets with different domains demonstrate that AdapTraj consistently outperforms other baselines by a substantial margin.

cs.AI↗

arXiv2023

Dynamic Frequency Domain Graph Convolutional Network for Traffic Forecasting

Complex spatial dependencies in transportation networks make traffic prediction extremely challenging. Much existing work is devoted to learning dynamic graph structures among sensors, and the strategy of mining spatial dependencies from traffic data, known as data-driven, tends to be an intuitive and effective approach. However, Time-Shift of traffic patterns and noise induced by random factors hinder data-driven spatial dependence modeling. In this paper, we propose a novel dynamic frequency domain graph convolution network (DFDGCN) to capture spatial dependencies. Specifically, we mitigate the effects of time-shift by Fourier transform, and introduce the identity embedding of sensors and time embedding when capturing data for graph learning since traffic data with noise is not entirely reliable. The graph is combined with static predefined and self-adaptive graphs during graph convolution to predict future traffic data through classical causal convolutions. Extensive experiments on four real-world datasets demonstrate that our model is effective and outperforms the baselines.

cs.LG↗

arXiv2023

Local Randomized Neural Networks with Hybridized Discontinuous Petrov-Galerkin Methods for Stokes-Darcy Flows

This paper introduces a new numerical approach that integrates local randomized neural networks (LRNNs) and the hybridized discontinuous Petrov-Galerkin (HDPG) method for solving coupled fluid flow problems. The proposed method partitions the domain of interest into several subdomains and constructs an LRNN on each subdomain. Then, the HDPG scheme is used to couple the LRNNs to approximate the unknown functions. We develop LRNN-HDPG methods based on velocity-stress formulation to solve two types of problems: Stokes-Darcy problems and Brinkman equations, which model the flow in porous media and free flow. We devise a simple and effective way to deal with the interface conditions in the Stokes-Darcy problems without adding extra terms to the numerical scheme. We conduct extensive numerical experiments to demonstrate the stability, efficiency, and robustness of the proposed method. The numerical results show that the LRNN-HDPG method can achieve high accuracy with a small number of degrees of freedom.

math.NA↗