arXiv subjects

Yang Yang

Publications and source records attributed to Yang Yang.

At least 775 records · Page 43Linked to original sources

arXiv2023

Two-Stage Coded Distributed Edge Learning: A Dynamic Partial Gradient Coding Perspective

The widespread adoption of distributed learning to train a global model from local data has been hindered by the challenge posed by stragglers. Recent attempts to mitigate this issue through gradient coding have proved difficult due to the large amounts of data redundancy, computational and communicational overhead it brings. Additionally, the complexity of encoding and decoding increases linearly with the number of local workers. In this paper, we present a lightweight coding method for the computing phase and a fair transmission protocol for the communication phase, to mitigate the straggler problem. A two-stage dynamic coding scheme is proposed for the computing phase, where partial gradients are computed by a portion of workers in the first stage and the remainder are decided based on their completion status in the first stage. To ensure fair communication, a perturbed Lyapunov function is designed to balance admission data fairness and maximize throughput. Extensive experimental results demonstrate the superiority of our proposed solution in terms of accuracy and resource utilization in the distributed learning system, even under practical network conditions and benchmark data.

cs.NI↗

arXiv2023

Collective magnetic Higgs excitation in a pyrochlore ruthenate

The emergence of scalar Higgs-type amplitude modes in systems where symmetry is spontaneously broken has been a highly successful, paradigmatic description of phase transitions, with implications ranging from high-energy particle physics to low-energy condensed matter systems. Here, we uncover two successive high temperature phase transitions in the pyrochlore magnet Nd$_2$Ru$_2$O$_7$ at $T_{\mathrm{N}} = 147$ K and $T^* = 97$ K, that lead to giant phonon instabilities and culminate in the emergence of a highly coherent excitation. This coherent excitation, distinct from other phonons and from conventional magnetic modes, stabilizes at a low energy of 3 meV. We assign it to a collective Higgs-type amplitude mode, that involves bond energy modulations of the Ru$_4$ tetrahedra. Its striking two-fold symmetry, incompatible with the underlying crystal structure, highlights the possibility of multiple entangled broken symmetries.

cond-mat.str-el↗

arXiv2023

Optimal control of infinite-dimensional differential systems with randomness and path-dependence and stochastic path-dependent Hamilton-Jacobi equations

This paper is devoted to the stochastic optimal control problem of infinite-dimensional differential systems allowing for both path-dependence and measurable randomness. As opposed to the deterministic path-dependent cases studied by Bayraktar and Keller [J. Funct. Anal. 275 (2018), 2096--2161], the value function turns out to be a random field on the path space and it is characterized by a stochastic path-dependent Hamilton-Jacobi (SPHJ) equation. A notion of viscosity solution is proposed and the value function is proved to be the unique viscosity solution to the associated SPHJ equation.

math.OC↗

arXiv2023

Maximizing Metapopulation Growth Rate and Biomass in Stream Networks

We consider the logistic metapopulation model over a stream network and use the metapopulation growth rate and the total biomass (of the positive equilibrium) as metrics for different aspects of population persistence. Our objective is to find distributions of resources that maximize these persistence measures. We begin our study by considering stream networks consisting of three nodes and prove that the strategy to maximize the total biomass is to concentrate all the resources in the most upstream locations. In contrast, when the diffusion rates are sufficiently small, the metapopulation growth rate is maximized when all resources are concentrated in one of the most downstream locations. These two main results are generalized to stream networks with any number of patches.

math.DS↗

arXiv2023

Cross-Domain Car Detection Model with Integrated Convolutional Block Attention Mechanism

Car detection, particularly through camera vision, has become a major focus in the field of computer vision and has gained widespread adoption. While current car detection systems are capable of good detection, reliable detection can still be challenging due to factors such as proximity between the car, light intensity, and environmental visibility. To address these issues, we propose cross-domain Car Detection Model with integrated convolutional block Attention mechanism(CDMA) that we apply to car recognition for autonomous driving and other areas. CDMA includes several novelties: 1)Building a complete cross-domain target detection framework. 2)Developing an unpaired target domain picture generation module with an integrated convolutional attention mechanism which specifically emphasizes the car headlights feature. 3)Adopting Generalized Intersection over Union (GIOU) as the loss function of the target detection framework. 4)Designing an object detection model integrated with two-headed Convolutional Block Attention Module(CBAM). 5)Utilizing an effective data enhancement method. To evaluate the model's effectiveness, we performed a reduced will resolution process on the data in the SSLAD dataset and used it as the benchmark dataset for our task. Experimental results show that the performance of the cross-domain car target detection model improves by 40% over the model without our framework, and our improvements have a significant impact on cross-domain car recognition.

cs.CV↗

arXiv2023

Towards KAB2S: Learning Key Knowledge from Single-Objective Problems to Multi-Objective Problem

As "a new frontier in evolutionary computation research", evolutionary transfer optimization(ETO) will overcome the traditional paradigm of zero reuse of related experience and knowledge from solved past problems in researches of evolutionary computation. In scheduling applications via ETO, a quite appealing and highly competitive framework "meeting" between them could be formed for both intelligent scheduling and green scheduling, especially for international pledge of "carbon neutrality" from China. To the best of our knowledge, our paper on scheduling here, serves as the 1st work of a class of ETO frameworks when multiobjective optimization problem "meets" single-objective optimization problems in discrete case (not multitasking optimization). More specifically, key knowledge conveyed for industrial applications, like positional building blocks with genetic algorithm based settings, could be used via the new core transfer mechanism and learning techniques for permutation flow shop scheduling problem(PFSP). Extensive studies on well-studied benchmarks validate firm effectiveness and great universality of our proposed ETO-PFSP framework empirically. Our investigations (1) enrich the ETO frameworks, (2) contribute to the classical and fundamental theory of building block for genetic algorithms and memetic algorithms, and (3) head towards the paradigm shift of evolutionary scheduling via learning by proposal and practice of paradigm of "knowledge and building-block based scheduling" (KAB2S) for "industrial intelligence" in China.

cs.NE↗

arXiv2023

Focusing on Relevant Responses for Multi-modal Rumor Detection

In the absence of an authoritative statement about a rumor, people may expose the truth behind such rumor through their responses on social media. Most rumor detection methods aggregate the information of all the responses and have made great progress. However, due to the different backgrounds of users, the responses have different relevance for discovering th suspicious points hidden in a rumor claim. The methods that focus on all the responding tweets would dilute the effect of the critical ones. Moreover, for a multi-modal rumor claim, the focus of a user may be on several words in the text or an object in the image, so the different modalities should be considered to select the relevant responses and verify the claim. In this paper, we propose a novel multi-modal rumor detection model, termed Focal Reasoning Model (FoRM), to filter out the irrelevant responses and further conduct fine-grained reasoning with the multi-modal claim and corresponding responses. Concretely, there are two main components in our FoRM: the coarse-grained selection and the fine-grained reasoning. The coarse-grained selection component leverages the post-level features of the responses to verify the claim and learns a relevant score of each response. Based on the relevant scores, the most relevant responses are reserved as the critical ones to the further reasoning. In the fine-grained reasoning component, we design a relation attention module to explore the fine-grained relations, i.e., token-to-token and token-to-object relations, between the reserved responses and the multi-modal claim for finding out the valuable clues. Extensive experiments have been conducted on two real-world datasets, and the results demonstrate that our proposed model outperforms all the baselines.

cs.SI↗

arXiv2023

BrainNet: Epileptic Wave Detection from SEEG with Hierarchical Graph Diffusion Learning

Epilepsy is one of the most serious neurological diseases, affecting 1-2% of the world's population. The diagnosis of epilepsy depends heavily on the recognition of epileptic waves, i.e., disordered electrical brainwave activity in the patient's brain. Existing works have begun to employ machine learning models to detect epileptic waves via cortical electroencephalogram (EEG). However, the recently developed stereoelectrocorticography (SEEG) method provides information in stereo that is more precise than conventional EEG, and has been broadly applied in clinical practice. Therefore, we propose the first data-driven study to detect epileptic waves in a real-world SEEG dataset. While offering new opportunities, SEEG also poses several challenges. In clinical practice, epileptic wave activities are considered to propagate between different regions in the brain. These propagation paths, also known as the epileptogenic network, are deemed to be a key factor in the context of epilepsy surgery. However, the question of how to extract an exact epileptogenic network for each patient remains an open problem in the field of neuroscience. To address these challenges, we propose a novel model (BrainNet) that jointly learns the dynamic diffusion graphs and models the brain wave diffusion patterns. In addition, our model effectively aids in resisting label imbalance and severe noise by employing several self-supervised learning tasks and a hierarchical framework. By experimenting with the extensive real SEEG dataset obtained from multiple patients, we find that BrainNet outperforms several latest state-of-the-art baselines derived from time-series analysis.

eess.SP↗

arXiv2023

MBrain: A Multi-channel Self-Supervised Learning Framework for Brain Signals

Brain signals are important quantitative data for understanding physiological activities and diseases of human brain. Most existing studies pay attention to supervised learning methods, which, however, require high-cost clinical labels. In addition, the huge difference in the clinical patterns of brain signals measured by invasive (e.g., SEEG) and non-invasive (e.g., EEG) methods leads to the lack of a unified method. To handle the above issues, we propose to study the self-supervised learning (SSL) framework for brain signals that can be applied to pre-train either SEEG or EEG data. Intuitively, brain signals, generated by the firing of neurons, are transmitted among different connecting structures in human brain. Inspired by this, we propose MBrain to learn implicit spatial and temporal correlations between different channels (i.e., contacts of the electrode, corresponding to different brain areas) as the cornerstone for uniformly modeling different types of brain signals. Specifically, we represent the spatial correlation by a graph structure, which is built with proposed multi-channel CPC. We theoretically prove that optimizing the goal of multi-channel CPC can lead to a better predictive representation and apply the instantaneou-time-shift prediction task based on it. Then we capture the temporal correlation by designing the delayed-time-shift prediction task. Finally, replace-discriminative-learning task is proposed to preserve the characteristics of each channel. Extensive experiments of seizure detection on both EEG and SEEG large-scale real-world datasets demonstrate that our model outperforms several state-of-the-art time series SSL and unsupervised models, and has the ability to be deployed to clinical practice.

eess.SP↗

arXiv2023

Accelerating Dynamic Network Embedding with Billions of Parameter Updates to Milliseconds

Network embedding, a graph representation learning method illustrating network topology by mapping nodes into lower-dimension vectors, is challenging to accommodate the ever-changing dynamic graphs in practice. Existing research is mainly based on node-by-node embedding modifications, which falls into the dilemma of efficient calculation and accuracy. Observing that the embedding dimensions are usually much smaller than the number of nodes, we break this dilemma with a novel dynamic network embedding paradigm that rotates and scales the axes of embedding space instead of a node-by-node update. Specifically, we propose the Dynamic Adjacency Matrix Factorization (DAMF) algorithm, which achieves an efficient and accurate dynamic network embedding by rotating and scaling the coordinate system where the network embedding resides with no more than the number of edge modifications changes of node embeddings. Moreover, a dynamic Personalized PageRank is applied to the obtained network embeddings to enhance node embeddings and capture higher-order neighbor information dynamically. Experiments of node classification, link prediction, and graph reconstruction on different-sized dynamic graphs suggest that DAMF advances dynamic network embedding. Further, we unprecedentedly expand dynamic network embedding experiments to billion-edge graphs, where DAMF updates billion-level parameters in less than 10ms.

cs.SI↗

arXiv2023

Searching for Radio Outflows from M31* with VLBI Observations

As one of the nearest and most dormant supermassive black holes (SMBHs), M31* provides a rare but promising opportunity for studying the physics of black hole accretion and feedback at the quiescent state. Previous Karl G. Jansky Very Large Array (VLA) observations with an arcsec resolution have detected M31* as a compact radio source over centimeter wavelengths, but the steep radio spectrum suggests optically-thin synchrotron radiation from an outflow driven by a hot accretion flow onto the SMBH. Aiming to probe the putative radio outflow, we have conducted milli-arcsec-resolution very long baseline interferometric (VLBI) observations of M31* in 2016, primarily at 5 GHz and combining the Very Long Baseline Array, Tianma-65m and Shanghai-25m Radio Telescopes. Despite the unprecedented simultaneous resolution and sensitivity achieved, no significant ($\gtrsim 3σ$) signal is detected at the putative position of M31* given an RMS level of $\rm 5.9~μJy\ beam^{-1}$, thus ruling out a point-like source with a peak flux density comparable to that ($\sim30~μJy\ beam^{-1}$) measured by the VLA observations taken in 2012. We disfavor the possibility that M31* has substantially faded since 2012, in view that a 2017 VLA observation successfully detected M31* at a historically-high peak flux density ($\sim75~μJy\ beam^{-1}$ at 6 GHz). Instead, the non-detection of the VLBI observations is best interpreted as the arcsec-scale core being resolved out at the milli-arcsec-scale, suggesting an intrinsic size of M31* at 5 GHz larger than $\sim300$ times the Schwarzschild radius. Such extended radio emission may originate from a hot wind driven by the weakly accreting SMBH.

astro-ph.GA↗

arXiv2023

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model

Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment on various devices. However, the deployment of knowledge distillation systems faces great challenges in real-world industrial-strength applications, which require the use of complex distillation methods on even larger-scale PLMs (over 10B), limited by memory on GPUs and the switching of methods. To overcome these challenges, we propose GKD, a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods. With GKD, developers can build larger distillation models on memory-limited GPUs and easily switch and combine different distillation methods within a single framework. Experimental results show that GKD can support the distillation of at least 100B-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs.

cs.CL↗

arXiv2023

DGraph: A Large-Scale Financial Dataset for Graph Anomaly Detection

Graph Anomaly Detection (GAD) has recently become a hot research spot due to its practicability and theoretical value. Since GAD emphasizes the application and the rarity of anomalous samples, enriching the varieties of its datasets is fundamental work. Thus, this paper present DGraph, a real-world dynamic graph in the finance domain. DGraph overcomes many limitations of current GAD datasets. It contains about 3M nodes, 4M dynamic edges, and 1M ground-truth nodes. We provide a comprehensive observation of DGraph, revealing that anomalous nodes and normal nodes generally have different structures, neighbor distribution, and temporal dynamics. Moreover, it suggests that unlabeled nodes are also essential for detecting fraudsters. Furthermore, we conduct extensive experiments on DGraph. Observation and experiments demonstrate that DGraph is propulsive to advance GAD research and enable in-depth exploration of anomalous nodes.

cs.SI↗

arXiv2023

A Novel Correlation-optimized Deep Learning Method for Wind Speed Forecast

The increasing installation rate of wind power poses great challenges to the global power system. In order to ensure the reliable operation of the power system, it is necessary to accurately forecast the wind speed and power of the wind turbines. At present, deep learning is progressively applied to the wind speed prediction. Nevertheless, the recent deep learning methods still reflect the embarrassment for practical applications due to model interpretability and hardware limitation. To this end, a novel deep knowledge-based learning method is proposed in this paper. The proposed method hybridizes pre-training method and auto-encoder structure to improve data representation and modeling of the deep knowledge-based learning framework. In order to form knowledge and corresponding absorbers, the original data is preprocessed by an optimization model based on correlation to construct multi-layer networks (knowledge) which are absorbed by sequence to sequence (Seq2Seq) models. Specifically, new cognition and memory units (CMU) are designed to reinforce traditional deep learning framework. Finally, the effectiveness of the proposed method is verified by three wind prediction cases from a wind farm in Liaoning, China. Experimental results show that the proposed method increases the stability and training efficiency compared to the traditional LSTM method and LSTM/GRU-based Seq2Seq method for applications of wind speed forecasting.

cs.LG↗

arXiv2023

When to Pre-Train Graph Neural Networks? From Data Generation Perspective!

In recent years, graph pre-training has gained significant attention, focusing on acquiring transferable knowledge from unlabeled graph data to improve downstream performance. Despite these recent endeavors, the problem of negative transfer remains a major concern when utilizing graph pre-trained models to downstream tasks. Previous studies made great efforts on the issue of what to pre-train and how to pre-train by designing a variety of graph pre-training and fine-tuning strategies. However, there are cases where even the most advanced "pre-train and fine-tune" paradigms fail to yield distinct benefits. This paper introduces a generic framework W2PGNN to answer the crucial question of when to pre-train (i.e., in what situations could we take advantage of graph pre-training) before performing effortful pre-training or fine-tuning. We start from a new perspective to explore the complex generative mechanisms from the pre-training data to downstream data. In particular, W2PGNN first fits the pre-training data into graphon bases, each element of graphon basis (i.e., a graphon) identifies a fundamental transferable pattern shared by a collection of pre-training graphs. All convex combinations of graphon bases give rise to a generator space, from which graphs generated form the solution space for those downstream data that can benefit from pre-training. In this manner, the feasibility of pre-training can be quantified as the generation probability of the downstream data from any generator in the generator space. W2PGNN offers three broad applications: providing the application scope of graph pre-trained models, quantifying the feasibility of pre-training, and assistance in selecting pre-training data to enhance downstream performance. We provide a theoretically sound solution for the first application and extensive empirical justifications for the latter two applications.

cs.LG↗

arXiv2023

A Novel Black Box Process Quality Optimization Approach based on Hit Rate

Hit rate is a key performance metric in predicting process product quality in integrated industrial processes. It represents the percentage of products accepted by downstream processes within a controlled range of quality. However, optimizing hit rate is a non-convex and challenging problem. To address this issue, we propose a data-driven quasi-convex approach that combines factorial hidden Markov models, multitask elastic net, and quasi-convex optimization. Our approach converts the original non-convex problem into a set of convex feasible problems, achieving an optimal hit rate. We verify the convex optimization property and quasi-convex frontier through Monte Carlo simulations and real-world experiments in steel production. Results demonstrate that our approach outperforms classical models, improving hit rates by at least 41.11% and 31.01% on two real datasets. Furthermore, the quasi-convex frontier provides a reference explanation and visualization for the deterioration of solutions obtained by conventional models.

cs.LG↗

arXiv2023

InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language

We present an interactive visual framework named InternGPT, or iGPT for short. The framework integrates chatbots that have planning and reasoning capabilities, such as ChatGPT, with non-verbal instructions like pointing movements that enable users to directly manipulate images or videos on the screen. Pointing (including gestures, cursors, etc.) movements can provide more flexibility and precision in performing vision-centric tasks that require fine-grained control, editing, and generation of visual content. The name InternGPT stands for \textbf{inter}action, \textbf{n}onverbal, and \textbf{chat}bots. Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2. Additionally, in iGPT, an auxiliary control mechanism is used to improve the control capability of LLM, and a large vision-language model termed Husky is fine-tuned for high-quality multi-modal dialogue (impressing ChatGPT-3.5-turbo with 93.89\% GPT-4 Quality). We hope this work can spark new ideas and directions for future interactive visual systems. Welcome to watch the code at https://github.com/OpenGVLab/InternGPT.

cs.CV↗

arXiv2023

IDToolkit: A Toolkit for Benchmarking and Developing Inverse Design Algorithms in Nanophotonics

Aiding humans with scientific designs is one of the most exciting of artificial intelligence (AI) and machine learning (ML), due to their potential for the discovery of new drugs, design of new materials and chemical compounds, etc. However, scientific design typically requires complex domain knowledge that is not familiar to AI researchers. Further, scientific studies involve professional skills to perform experiments and evaluations. These obstacles prevent AI researchers from developing specialized methods for scientific designs. To take a step towards easy-to-understand and reproducible research of scientific design, we propose a benchmark for the inverse design of nanophotonic devices, which can be verified computationally and accurately. Specifically, we implemented three different nanophotonic design problems, namely a radiative cooler, a selective emitter for thermophotovoltaics, and structural color filters, all of which are different in design parameter spaces, complexity, and design targets. The benchmark environments are implemented with an open-source simulator. We further implemented 10 different inverse design algorithms and compared them in a reproducible and fair framework. The results revealed the strengths and weaknesses of existing methods, which shed light on several future directions for developing more efficient inverse design algorithms. Our benchmark can also serve as the starting point for more challenging scientific design problems. The code of IDToolkit is available at https://github.com/ThyrixYang/IDToolkit.

cs.AI↗