arXiv subjects

Chao Wu

Publications and source records attributed to Chao Wu.

At least 109 records · Page 6Linked to original sources

arXiv2024

From Chaos to Clarity: Time Series Anomaly Detection in Astronomical Observations

With the development of astronomical facilities, large-scale time series data observed by these facilities is being collected. Analyzing anomalies in these astronomical observations is crucial for uncovering potential celestial events and physical phenomena, thus advancing the scientific research process. However, existing time series anomaly detection methods fall short in tackling the unique characteristics of astronomical observations where each star is inherently independent but interfered by random concurrent noise, resulting in a high rate of false alarms. To overcome the challenges, we propose AERO, a novel two-stage framework tailored for unsupervised anomaly detection in astronomical observations. In the first stage, we employ a Transformer-based encoder-decoder architecture to learn the normal temporal patterns on each variate (i.e., star) in alignment with the characteristic of variate independence. In the second stage, we enhance the graph neural network with a window-wise graph structure learning to tackle the occurrence of concurrent noise characterized by spatial and temporal randomness. In this way, AERO is not only capable of distinguishing normal temporal patterns from potential anomalies but also effectively differentiating concurrent noise, thus decreasing the number of false alarms. We conducted extensive experiments on three synthetic datasets and three real-world datasets. The results demonstrate that AERO outperforms the compared baselines. Notably, compared to the state-of-the-art model, AERO improves the F1-score by up to 8.76% and 2.63% on synthetic and real-world datasets respectively.

cs.LG↗

arXiv2024

Generalizable Two-Branch Framework for Image Class-Incremental Learning

Deep neural networks often severely forget previously learned knowledge when learning new knowledge. Various continual learning (CL) methods have been proposed to handle such a catastrophic forgetting issue from different perspectives and achieved substantial improvements. In this paper, a novel two-branch continual learning framework is proposed to further enhance most existing CL methods. Specifically, the main branch can be any existing CL model and the newly introduced side branch is a lightweight convolutional network. The output of each main branch block is modulated by the output of the corresponding side branch block. Such a simple two-branch model can then be easily implemented and learned with the vanilla optimization setting without whistles and bells. Extensive experiments with various settings on multiple image datasets show that the proposed framework yields consistent improvements over state-of-the-art methods.

cs.CV↗

arXiv2024

Coexisting Magnetism, Ferroelectric, and Ferrovalley Multiferroic in Stacking-Dependent Two-Dimensional Materials

The two-dimensional (2D) multiferroic materials have widespread of application prospects in facilitating the integration and miniaturization of nanodevices. However, it is rarely coupling between the magnetic, ferroelectric, and ferrovalley in one 2D material. Here, we propose a mechanism for manipulating magnetism, ferroelectric, and valley polarization by interlayer sliding in 2D bilayer material. Monolayer GdI2 exhibits a ferromagnetic semiconductor with the valley polarization up to 155.5 meV. More interestingly, the magnetism and valley polarization of bilayer GdI2 can be strongly coupled by sliding ferroelectricity, appearing these tunable and reversible. In addition, we uncover the microscopic mechanism of magnetic phase transition by spin Hamiltonian and electron hopping between layers. Our findings offer a new direction for investigating 2D multiferroic in the implication for next-generation electronic, valleytronic, and spintronic devices.

cond-mat.mtrl-sci↗

arXiv2024

Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models

Catastrophic forgetting emerges as a critical challenge when fine-tuning multi-modal large language models (MLLMs), where improving performance on unseen tasks often leads to a significant performance drop on the original tasks. This paper presents a comprehensive analysis of catastrophic forgetting in MLLMs and introduces a post-training adjustment method called Model Tailor. Our method primarily preserves the pre-trained parameters while replacing a small number ($\leq$ 10\%) of fine-tuned parameters, maintaining $\sim$ 99\% effectiveness on original tasks versus pre-training, and achieving $\sim$ 97\% on new tasks compared to standard fine-tuning. Specifically, we derive a sparse mask to identify the "model patch", based on a fusion strategy that integrates salience and sensitivity analysis. Subsequently, a compensation mechanism is introduced to "decorate the patch", enhancing the model's performance on both target and original tasks. Additionally, our method is adaptable to multi-task scenarios. Through extensive experiments on InstructBLIP and LLaVA-1.5 in both image captioning and visual question answering tasks, our approach demonstrates significant task adaptability while preserving inherent pre-trained capabilities.

cs.CL↗

arXiv2024

Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers

Geometric fracture assembly presents a challenging practical task in archaeology and 3D computer vision. Previous methods have focused solely on assembling fragments based on semantic information, which has limited the quantity of objects that can be effectively assembled. Therefore, there is a need to develop a scalable framework for geometric fracture assembly without relying on semantic information. To improve the effectiveness of assembling geometric fractures without semantic information, we propose a co-creation space comprising several assemblers capable of gradually and unambiguously assembling fractures. Additionally, we introduce a novel loss function, i.e., the geometric-based collision loss, to address collision issues during the fracture assembly process and enhance the results. Our framework exhibits better performance on both PartNet and Breaking Bad datasets compared to existing state-of-the-art frameworks. Extensive experiments and quantitative comparisons demonstrate the effectiveness of our proposed framework, which features linear computational complexity, enhanced abstraction, and improved generalization. Our code is publicly available at https://github.com/Ruiyuan-Zhang/CCS.

cs.CV↗

arXiv2023

Exploring $HVV$ amplitudes with $CP$ violation by decomposition and on-shell scattering amplitude methods

$CP$ violation may play an important role in Baryogenesis in early universe, and should be examined at colliders comprehensively. We study $CP$ properties of $HVV$ vertexes between Higgs and gauge boson pairs with defining a $CP$ violation phase angle $ξ$, which indicates the mixture of $CP$-even and $CP$-odd Higgs states in $HVV$ in new physics. A series of $HVV$ amplitudes $H\toγγ, H\toγV\to γ\ell\ell$, and $H\to VV\to 4\ell$ with $CP$ phase angle are studied systematically, which explains explicitly why $CP$ violation could only be probed in $4\ell$ process independently. We get a novel amplitude decomposition relation which illustrates if two preconditions (multilinear momentum dependent vertexes and current $J_μ$ of $V\to \ell^+ \ell^-$ is formally proportional to a photon's polarization vector) are satisfied, an high-point amplitude can be decomposed into a summation of a series of low-point amplitudes. As a practical example, the amplitude of $H\toγV\to γ\ell\ell$, and $H\to VV\to 4\ell$ processes can be decomposed into summation of many $H\toγγ$ amplitudes. Meanwhile, we calculate these amplitudes in the framework of on-shell scattering amplitude method, with considering both massless and massive vector gauge bosons with $CP$ violation phase angle. The above two approaches provides consistent results and exhibit clearly the $CP$ violation $ξ$ dependence in the amplitudes.

hep-ph↗

arXiv2023

FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning

Personalized federated learning (pFL) enables collaborative training among multiple clients to enhance the capability of customized local models. In pFL, clients may have heterogeneous (also known as non-IID) data, which poses a key challenge in how to decouple the data knowledge into generic knowledge for global sharing and personalized knowledge for preserving local personalization. A typical way of pFL focuses on label distribution skew, and they adopt a decoupling scheme where the model is split into a common feature extractor and two prediction heads (generic and personalized). However, such a decoupling scheme cannot solve the essential problem of feature skew heterogeneity, because a common feature extractor cannot decouple the generic and personalized features. Therefore, in this paper, we rethink the architecture decoupling design for feature-skew pFL and propose an effective pFL method called FediOS. In FediOS, we reformulate the decoupling into two feature extractors (generic and personalized) and one shared prediction head. Orthogonal projections are used for clients to map the generic features into one common subspace and scatter the personalized features into different subspaces to achieve decoupling for them. In addition, a shared prediction head is trained to balance the importance of generic and personalized features during inference. Extensive experiments on four vision datasets demonstrate our method reaches state-of-the-art pFL performances under feature skew heterogeneity.

cs.LG↗

arXiv2023

Multifield tunable valley splitting in two-dimensional MXene Cr$_2$COOH

Manipulation of the valley degree of freedom provides a novel paradigm in quantum information technology. Here, through first-principles calculations and model analysis, we demonstrate that monolayer Cr$_2$COOH MXene is a promising candidate material for valleytronics applications. We reveal that Cr$_2$COOH is a ferromagnetic semiconductor and harbors valley features. Due to the simultaneous breaking inversion symmetry and time-reversal symmetry, the valleys are polarized spontaneously. Moreover, the valley polarization is sizeable in both the valence and conduction bands, benefiting the observation of the anomalous valley Hall effect. More remarkably, the valley splitting can be effectively tuned by the magnetization direction, strain and ferroelectric substrate. More interestingly, the ferroelectric substrate Sc$_2$CO$_2$ can not only regulate the MAE, but also tune valley polarization state. Our findings offer a practical way for realizing highly tunable valleys by multiferroic couplings.

cond-mat.mtrl-sci↗

arXiv2023

Score-PA: Score-based 3D Part Assembly

Autonomous 3D part assembly is a challenging task in the areas of robotics and 3D computer vision. This task aims to assemble individual components into a complete shape without relying on predefined instructions. In this paper, we formulate this task from a novel generative perspective, introducing the Score-based 3D Part Assembly framework (Score-PA) for 3D part assembly. Knowing that score-based methods are typically time-consuming during the inference stage. To address this issue, we introduce a novel algorithm called the Fast Predictor-Corrector Sampler (FPC) that accelerates the sampling process within the framework. We employ various metrics to assess assembly quality and diversity, and our evaluation results demonstrate that our algorithm outperforms existing state-of-the-art approaches. We release our code at https://github.com/J-F-Cheng/Score-PA_Score-based-3D-Part-Assembly.

cs.CV↗

arXiv2023

Understanding Prompt Tuning for V-L Models Through the Lens of Neural Collapse

Large-scale vision-language (V-L) models have demonstrated remarkable generalization capabilities for downstream tasks through prompt tuning. However, the mechanisms behind the learned text representations are unknown, limiting further generalization gains, especially under class imbalance scenarios. Recent advances in the neural collapse (NC) phenomenon of vision-only models suggest that the optimal representation structure is the simplex ETF, which paves the way to study representations in V-L models. In this paper, we make the first attempt to use NC for examining the representations in V-L models via prompt tuning. It is found that NC optimality of text-to-image representations shows a positive correlation with downstream generalizability, which is more severe under class imbalance settings. To improve the representations, we propose Neural-collapse-anchored Prompt Tuning (NPT), a novel method that learns prompts with text and image representations that satisfy the same simplex ETF. NPT incorporates two regularization terms: language-modality collapse and multi-modality isomorphism; and it is compatible with other prompt tuning methods. Extensive experiments show that NPT can consistently help to improve existing prompt tuning techniques across 11 datasets for both balanced and imbalanced settings.

cs.CV↗

arXiv2023

Universal Domain Adaptation via Compressive Attention Matching

Universal domain adaptation (UniDA) aims to transfer knowledge from the source domain to the target domain without any prior knowledge about the label set. The challenge lies in how to determine whether the target samples belong to common categories. The mainstream methods make judgments based on the sample features, which overemphasizes global information while ignoring the most crucial local objects in the image, resulting in limited accuracy. To address this issue, we propose a Universal Attention Matching (UniAM) framework by exploiting the self-attention mechanism in vision transformer to capture the crucial object information. The proposed framework introduces a novel Compressive Attention Matching (CAM) approach to explore the core information by compressively representing attentions. Furthermore, CAM incorporates a residual-based measurement to determine the sample commonness. By utilizing the measurement, UniAM achieves domain-wise and category-wise Common Feature Alignment (CFA) and Target Class Separation (TCS). Notably, UniAM is the first method utilizing the attention in vision transformer directly to perform classification tasks. Extensive experiments show that UniAM outperforms the current state-of-the-art methods on various benchmark datasets.

cs.CV↗

arXiv2023

Generalized Universal Domain Adaptation with Generative Flow Networks

We introduce a new problem in unsupervised domain adaptation, termed as Generalized Universal Domain Adaptation (GUDA), which aims to achieve precise prediction of all target labels including unknown categories. GUDA bridges the gap between label distribution shift-based and label space mismatch-based variants, essentially categorizing them as a unified problem, guiding to a comprehensive framework for thoroughly solving all the variants. The key challenge of GUDA is developing and identifying novel target categories while estimating the target label distribution. To address this problem, we take advantage of the powerful exploration capability of generative flow networks and propose an active domain adaptation algorithm named GFlowDA, which selects diverse samples with probabilities proportional to a reward function. To enhance the exploration capability and effectively perceive the target label distribution, we tailor the states and rewards, and introduce an efficient solution for parent exploration and state transition. We also propose a training paradigm for GUDA called Generalized Universal Adversarial Network (GUAN), which involves collaborative optimization between GUAN and GFlowNet. Theoretical analysis highlights the importance of exploration, and extensive experiments on benchmark datasets demonstrate the superiority of GFlowDA.

cs.CV↗

arXiv2023

No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier

Data heterogeneity is an inherent challenge that hinders the performance of federated learning (FL). Recent studies have identified the biased classifiers of local models as the key bottleneck. Previous attempts have used classifier calibration after FL training, but this approach falls short in improving the poor feature representations caused by training-time classifier biases. Resolving the classifier bias dilemma in FL requires a full understanding of the mechanisms behind the classifier. Recent advances in neural collapse have shown that the classifiers and feature prototypes under perfect training scenarios collapse into an optimal structure called simplex equiangular tight frame (ETF). Building on this neural collapse insight, we propose a solution to the FL's classifier bias problem by utilizing a synthetic and fixed ETF classifier during training. The optimal classifier structure enables all clients to learn unified and optimal feature representations even under extremely heterogeneous data. We devise several effective modules to better adapt the ETF structure in FL, achieving both high generalization and personalization. Extensive experiments demonstrate that our method achieves state-of-the-art performances on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

cs.LG↗

arXiv2023

Using Adamic-Adar Index Algorithm to Predict Volunteer Collaboration: Less is More

Social networks exhibit a complex graph-like structure due to the uncertainty surrounding potential collaborations among participants. Machine learning algorithms possess generic outstanding performance in multiple real-world prediction tasks. However, whether machine learning algorithms outperform specific algorithms designed for graph link prediction remains unknown to us. To address this issue, the Adamic-Adar Index (AAI), Jaccard Coefficient (JC) and common neighbour centrality (CNC) as representatives of graph-specific algorithms were applied to predict potential collaborations, utilizing data from volunteer activities during the Covid-19 pandemic in Shenzhen city, along with the classical machine learning algorithms such as random forest, support vector machine, and gradient boosting as single predictors and components of ensemble learning. This paper introduces that the AAI algorithm outperformed the traditional JC and CNC, and other machine learning algorithms in analyzing graph node attributes for this task.

cs.SI↗

arXiv2023

White-light superflare and long-term activity of the nearby M7 type binary EI~Cnc observed with GWAC system

Stellar white-light flares are believed to play an essential role on the physical and chemical properties of the atmosphere of the surrounding exoplanets. Here we report an optical monitoring campaign on the nearby flaring system EI~Cnc carried out by the Ground-based Wide Angle Cameras (GWAC) and its dedicated follow-up telescope. A superflare, coming from the brighter component EI~CncA, was detected and observed, in which four components are required to properly model the complex decay light curve. The lower limit of flare energy in the $R-$band is estimated to be $3.3\times10^{32}$ ergs. 27 flares are additionally detected from the GWAC archive data with a total duration of 290 hours. The inferred cumulative flare frequency distribution follows a quite shallow power-law function with a slope of $β=-0.50\pm 0.03$ over the energy range between $10^{30}$ and $10^{33}$ erg, which reinforces the trend that stars cooler than M4 show enhanced superflare activity. The flares identified in EI~Cnc enable us to extend the $τ-E$ relationship previously established in the white-light superflares of solar-type stars down to an energy as low as $\sim10^{30}$erg (i.e., by three orders): $τ\propto E^{0.42\pm0.02}$, which suggests a common flare mechanism for stars with a type from M to solar-like, and implies an invariant of $B^{1/3}\upsilon_{\rm A}$ in the white-light flares.

astro-ph.SR↗

arXiv2023

Revisiting Weighted Aggregation in Federated Learning with Neural Networks

In federated learning (FL), weighted aggregation of local models is conducted to generate a global model, and the aggregation weights are normalized (the sum of weights is 1) and proportional to the local data sizes. In this paper, we revisit the weighted aggregation process and gain new insights into the training dynamics of FL. First, we find that the sum of weights can be smaller than 1, causing global weight shrinking effect (analogous to weight decay) and improving generalization. We explore how the optimal shrinking factor is affected by clients' data heterogeneity and local epochs. Second, we dive into the relative aggregation weights among clients to depict the clients' importance. We develop client coherence to study the learning dynamics and find a critical point that exists. Before entering the critical point, more coherent clients play more essential roles in generalization. Based on the above insights, we propose an effective method for Federated Learning with Learnable Aggregation Weights, named as FedLAW. Extensive experiments verify that our method can improve the generalization of the global model by a large margin on different datasets and models.

cs.LG↗

arXiv2023

Data valuation: The partial ordinal Shapley value for machine learning

Data valuation using Shapley value has emerged as a prevalent research domain in machine learning applications. However, it is a challenge to address the role of order in data cooperation as most research lacks such discussion. To tackle this problem, this paper studies the definition of the partial ordinal Shapley value by group theory in abstract algebra. Besides, since the calculation of the partial ordinal Shapley value requires exponential time, this paper also gives three algorithms for approximating the results. The Truncated Monte Carlo algorithm is derived from the classic Shapley value approximation algorithm. The Classification Monte Carlo algorithm and the Classification Truncated Monte Carlo algorithm are based on the fact that the data points in the same class provide similar information, then we can accelerate the calculation by leaving out some data points in each class.

cs.LG↗

arXiv2023

Edge-cloud Collaborative Learning with Federated and Centralized Features

Federated learning (FL) is a popular way of edge computing that doesn't compromise users' privacy. Current FL paradigms assume that data only resides on the edge, while cloud servers only perform model averaging. However, in real-life situations such as recommender systems, the cloud server has the ability to store historical and interactive features. In this paper, our proposed Edge-Cloud Collaborative Knowledge Transfer Framework (ECCT) bridges the gap between the edge and cloud, enabling bi-directional knowledge transfer between both, sharing feature embeddings and prediction logits. ECCT consolidates various benefits, including enhancing personalization, enabling model heterogeneity, tolerating training asynchronization, and relieving communication burdens. Extensive experiments on public and industrial datasets demonstrate ECCT's effectiveness and potential for use in academia and industry.

cs.LG↗