arXiv subjects

Xiao Wang

Publications and source records attributed to Xiao Wang.

At least 631 records · Page 35Linked to original sources

arXiv2021

AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries

Contrastive learning relies on constructing a collection of negative examples that are sufficiently hard to discriminate against positive queries when their representations are self-trained. Existing contrastive learning methods either maintain a queue of negative samples over minibatches while only a small portion of them are updated in an iteration, or only use the other examples from the current minibatch as negatives. They could not closely track the change of the learned representation over iterations by updating the entire queue as a whole, or discard the useful information from the past minibatches. Alternatively, we present to directly learn a set of negative adversaries playing against the self-trained representation. Two players, the representation network and negative adversaries, are alternately updated to obtain the most challenging negative examples against which the representation of positive queries will be trained to discriminate. We further show that the negative adversaries are updated towards a weighted combination of positive queries by maximizing the adversarial contrastive loss, thereby allowing them to closely track the change of representations over time. Experiment results demonstrate the proposed Adversarial Contrastive (AdCo) model not only achieves superior performances (a top-1 accuracy of 73.2\% over 200 epochs and 75.7\% over 800 epochs with linear evaluation on ImageNet), but also can be pre-trained more efficiently with fewer epochs.

cs.LG↗

arXiv2021

Self-compression of stimulated Raman backscattering by flying focus

A novel regime of self-compression is proposed for plasma-based backward Raman amplification(BRA) upon flying focus. By using a pumping focus moving with a speed equal to the group velocity of stimulated Raman backscattering(SRBS), only a short part of SRBS which does always synchronize with the flying focus can be amplified. Due to the asymmetrical amplification, the pulse can be directly compressed in the linear stage of BRA. Therefore, instead of a short pulse, the Raman spontaneous or a long pulse can seed the BRA amplifiers. The regime is supported by the 2D particle-in-cell(PIC) simulation without a seed, presenting that the pump pulse is compressed from 26ps to 116fs, with an output amplitude comparable with the case of a well-synchronized short seed. This method provides a significant way to simplify the Raman amplifiers and overcome the issue of synchronization jitter between the pump and the seed.

physics.plasm-ph↗

arXiv2021

EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations

Deep neural networks have been successfully applied to many real-world applications. However, such successes rely heavily on large amounts of labeled data that is expensive to obtain. Recently, many methods for semi-supervised learning have been proposed and achieved excellent performance. In this study, we propose a new EnAET framework to further improve existing semi-supervised methods with self-supervised information. To our best knowledge, all current semi-supervised methods improve performance with prediction consistency and confidence ideas. We are the first to explore the role of {\bf self-supervised} representations in {\bf semi-supervised} learning under a rich family of transformations. Consequently, our framework can integrate the self-supervised information as a regularization term to further improve {\it all} current semi-supervised methods. In the experiments, we use MixMatch, which is the current state-of-the-art method on semi-supervised learning, as a baseline to test the proposed EnAET framework. Across different datasets, we adopt the same hyper-parameters, which greatly improves the generalization ability of the EnAET framework. Experiment results on different datasets demonstrate that the proposed EnAET framework greatly improves the performance of current semi-supervised algorithms. Moreover, this framework can also improve {\bf supervised learning} by a large margin, including the extremely challenging scenarios with only 10 images per class. The code and experiment records are available in \url{https://github.com/maple-research-lab/EnAET}.

cs.CV↗

arXiv2021

Interpreting and Unifying Graph Neural Networks with An Optimization Framework

Graph Neural Networks (GNNs) have received considerable attention on graph-structured data learning for a wide variety of tasks. The well-designed propagation mechanism which has been demonstrated effective is the most fundamental part of GNNs. Although most of GNNs basically follow a message passing manner, litter effort has been made to discover and analyze their essential relations. In this paper, we establish a surprising connection between different propagation mechanisms with a unified optimization problem, showing that despite the proliferation of various GNNs, in fact, their proposed propagation mechanisms are the optimal solution optimizing a feature fitting function over a wide class of graph kernels with a graph regularization term. Our proposed unified optimization framework, summarizing the commonalities between several of the most representative GNNs, not only provides a macroscopic view on surveying the relations between different GNNs, but also further opens up new opportunities for flexibly designing new GNNs. With the proposed framework, we discover that existing works usually utilize naive graph convolutional kernels for feature fitting function, and we further develop two novel objective functions considering adjustable graph kernels showing low-pass or high-pass filtering capabilities respectively. Moreover, we provide the convergence proofs and expressive power comparisons for the proposed models. Extensive experiments on benchmark datasets clearly show that the proposed GNNs not only outperform the state-of-the-art methods but also have good ability to alleviate over-smoothing, and further verify the feasibility for designing GNNs with our unified optimization framework.

cs.LG↗

arXiv2021

Heterogeneous Graph Attention Network

Graph neural network, as a powerful graph representation technique based on deep learning, has shown superior performance and attracted considerable research interest. However, it has not been fully considered in graph neural network for heterogeneous graph which contains different types of nodes and links. The heterogeneity and rich semantic information bring great challenges for designing a graph neural network for heterogeneous graph. Recently, one of the most exciting advancements in deep learning is the attention mechanism, whose great potential has been well demonstrated in various areas. In this paper, we first propose a novel heterogeneous graph neural network based on the hierarchical attention, including node-level and semantic-level attentions. Specifically, the node-level attention aims to learn the importance between a node and its metapath based neighbors, while the semantic-level attention is able to learn the importance of different meta-paths. With the learned importance from both node-level and semantic-level attention, the importance of node and meta-path can be fully considered. Then the proposed model can generate node embedding by aggregating features from meta-path based neighbors in a hierarchical manner. Extensive experimental results on three real-world heterogeneous graphs not only show the superior performance of our proposed model over the state-of-the-arts, but also demonstrate its potentially good interpretability for graph analysis.

cs.SI↗

arXiv2021

Beyond Low-frequency Information in Graph Convolutional Networks

Graph neural networks (GNNs) have been proven to be effective in various network-related tasks. Most existing GNNs usually exploit the low-frequency signals of node features, which gives rise to one fundamental question: is the low-frequency information all we need in the real world applications? In this paper, we first present an experimental investigation assessing the roles of low-frequency and high-frequency signals, where the results clearly show that exploring low-frequency signal only is distant from learning an effective node representation in different scenarios. How can we adaptively learn more information beyond low-frequency information in GNNs? A well-informed answer can help GNNs enhance the adaptability. We tackle this challenge and propose a novel Frequency Adaptation Graph Convolutional Networks (FAGCN) with a self-gating mechanism, which can adaptively integrate different signals in the process of message passing. For a deeper understanding, we theoretically analyze the roles of low-frequency signals and high-frequency signals on learning node representations, which further explains why FAGCN can perform well on different types of networks. Extensive experiments on six real-world networks validate that FAGCN not only alleviates the over-smoothing problem, but also has advantages over the state-of-the-arts.

cs.LG↗

arXiv2020

The Analog Front-end for the LGAD Based Precision Timing Application in CMS ETL

The analog front-end for the Low Gain Avalanche Detector (LGAD) based precision timing application in the CMS Endcap Timing Layer (ETL) has been prototyped in a 65 nm CMOS mini-ASIC named ETROC0. Serving as the very first prototype of ETL readout chip (ETROC), ETROC0 aims to study and demonstrate the performance of the analog frontend, with the goal to achieve 40 to 50 ps time resolution per hit with LGAD (therefore reach about 30ps per track with two detector-layer hits per track). ETROC0 consists of preamplifier and discriminator stages, which amplifies the LGAD signal and generates digital pulses containing time of arrival and time over threshold information. This paper will focus on the design considerations that lead to the ETROC front-end architecture choice, the key design features of the building blocks, the methodology of using the LGAD simulation data to evaluate and optimize the front-end design. The ETROC0 prototype chips have been extensively tested using charge injection and the measured performance agrees well with simulation. The initial beam test results are also presented, with time resolution of around 33 ps observed from the preamplifier waveform analysis and around 41 ps from the discriminator pulses analysis. A subset of ETROC0 chips have also been tested to a total ionizing dose of 100 MRad with X-ray and no performance degradation been observed.

physics.ins-det↗

arXiv2020

yNet: a multi-input convolutional network for ultra-fast simulation of field evolvement

The capability of multi-input field-to-field regression, i.e. mapping the initial field and applied conditions to the evolved field, is appealing, enabling ultra-fast physics-free simulation of various field evolvements across many disciplines. We hereby propose a y-shaped multi-input deep convolutional network, yNet, which can effectively account for combined effects of multiple mixed inputs on output field. The proposed yNet is applied to the simulation of porosity evolution in selective lasering sintering (SLS). Upon testing, yNet can simulate nearly identical porosity evolution and development to the physics-based model, with a 99.13% morphological similarity for various SLS conditions. We then effortlessly boost the porosity simulation capability to the realistic, full-component level. yNet is generally applicable to simulating various structural/morphological evolutions and other condition-concerned, continuous field evolvements even with spatially and/or temporally non-uniform evolving kinetics. Once trained, the light-weight yNet can be distributed easily and ran with limited computational resource while reducing computation time to a split second. It thus may have a transformative impact by democratizing the capability of ultra-fast and extreme-scale simulation of various field evolvements.

cs.CE↗

arXiv2020

High-Dimensional Spatial Quantile Function-on-Scalar Regression

This paper develops a novel spatial quantile function-on-scalar regression model, which studies the conditional spatial distribution of a high-dimensional functional response given scalar predictors. With the strength of both quantile regression and copula modeling, we are able to explicitly characterize the conditional distribution of the functional or image response on the whole spatial domain. Our method provides a comprehensive understanding of the effect of scalar covariates on functional responses across different quantile levels and also gives a practical way to generate new images for given covariate values. Theoretically, we establish the minimax rates of convergence for estimating coefficient functions under both fixed and random designs. We further develop an efficient primal-dual algorithm to handle high-dimensional image data. Simulations and real data analysis are conducted to examine the finite-sample performance.

stat.ME↗

arXiv2020

Addressing Class Imbalance in Federated Learning

Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients' side may lead to significant challenges such as class imbalance and non-IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or servers, and previous methods for class imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting class imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function -- \textbf{Ratio Loss} to mitigate the impact of the imbalance. Our experiments demonstrate the importance of acknowledging class imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.

cs.LG↗

arXiv2020

Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev

Sampling is a fundamental and arguably very important task with numerous applications in Machine Learning. One approach to sample from a high dimensional distribution $e^{-f}$ for some function $f$ is the Langevin Algorithm (LA). Recently, there has been a lot of progress in showing fast convergence of LA even in cases where $f$ is non-convex, notably [53], [39] in which the former paper focuses on functions $f$ defined in $\mathbb{R}^n$ and the latter paper focuses on functions with symmetries (like matrix completion type objectives) with manifold structure. Our work generalizes the results of [53] where $f$ is defined on a manifold $M$ rather than $\mathbb{R}^n$. From technical point of view, we show that KL decreases in a geometric rate whenever the distribution $e^{-f}$ satisfies a log-Sobolev inequality on $M$.

cs.LG↗

arXiv2020

A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources

Heterogeneous graphs (HGs) also known as heterogeneous information networks have become ubiquitous in real-world scenarios; therefore, HG embedding, which aims to learn representations in a lower-dimension space while preserving the heterogeneous structures and semantics for downstream tasks (e.g., node/graph classification, node clustering, link prediction), has drawn considerable attentions in recent years. In this survey, we perform a comprehensive review of the recent development on HG embedding methods and techniques. We first introduce the basic concepts of HG and discuss the unique challenges brought by the heterogeneity for HG embedding in comparison with homogeneous graph representation learning; and then we systemically survey and categorize the state-of-the-art HG embedding methods based on the information they used in the learning process to address the challenges posed by the HG heterogeneity. In particular, for each representative HG embedding method, we provide detailed introduction and further analyze its pros and cons; meanwhile, we also explore the transformativeness and applicability of different types of HG embedding methods in the real-world industrial environments for the first time. In addition, we further present several widely deployed systems that have demonstrated the success of HG embedding techniques in resolving real-world application problems with broader impacts. To facilitate future research and applications in this area, we also summarize the open-source code, existing graph learning platforms and benchmark datasets. Finally, we explore the additional issues and challenges of HG embedding and forecast the future research directions in this field.

cs.SI↗

arXiv2020

Bubble wall velocity beyond leading-log approximation in electroweak phase transition

The bubble wall velocity is essential for the phase transition dynamics in the early universe and its cosmological implications, such as the energy budget of phase transition gravitational wave and electroweak baryogenesis. One key factor to determine the wall velocity is the collision term that quantifies the interactions between the massive particles in the plasma and the bubble wall. We improve the calculations of the collision term beyond the leading-log approximation, and further obtain more precise bubble wall velocity for a representative effective model.

hep-ph↗

arXiv2020

Cross-Lingual Document Retrieval with Smooth Learning

Cross-lingual document search is an information retrieval task in which the queries' language differs from the documents' language. In this paper, we study the instability of neural document search models and propose a novel end-to-end robust framework that achieves improved performance in cross-lingual search with different documents' languages. This framework includes a novel measure of the relevance, smooth cosine similarity, between queries and documents, and a novel loss function, Smooth Ordinal Search Loss, as the objective. We further provide theoretical guarantee on the generalization error bound for the proposed framework. We conduct experiments to compare our approach with other document search models, and observe significant gains under commonly used ranking metrics on the cross-lingual document retrieval task in a variety of languages.

cs.IR↗

arXiv2020

Dynamic Fusion based Federated Learning for COVID-19 Detection

Medical diagnostic image analysis (e.g., CT scan or X-Ray) using machine learning is an efficient and accurate way to detect COVID-19 infections. However, sharing diagnostic images across medical institutions is usually not allowed due to the concern of patients' privacy. This causes the issue of insufficient datasets for training the image classification model. Federated learning is an emerging privacy-preserving machine learning paradigm that produces an unbiased global model based on the received updates of local models trained by clients without exchanging clients' local data. Nevertheless, the default setting of federated learning introduces huge communication cost of transferring model updates and can hardly ensure model performance when data heterogeneity of clients heavily exists. To improve communication efficiency and model performance, in this paper, we propose a novel dynamic fusion-based federated learning approach for medical diagnostic image analysis to detect COVID-19 infections. First, we design an architecture for dynamic fusion-based federated learning systems to analyse medical diagnostic images. Further, we present a dynamic fusion method to dynamically decide the participating clients according to their local model performance and schedule the model fusion-based on participating clients' training time. In addition, we summarise a category of medical diagnostic image datasets for COVID-19 detection, which can be used by the machine learning community for image analysis. The evaluation results show that the proposed approach is feasible and performs better than the default setting of federated learning in terms of model performance, communication efficiency and fault tolerance.

cs.DC↗

arXiv2020

Domain Priori Knowledge based Integrated Solution Design for Internet of Services

Various types of services, such as web APIs, IoT services, O2O services, and many others, have flooded on the Internet. Interconnections among these services have resulted in a new phenomenon called "Internet of Services" (IoS). By IoS,people don't need to request multiple services by themselves to fulfill their daily requirements, but it is an IoS platform that is responsible for constructing integrated solutions for them. Since user requirements (URs) are usually coarse-grained and transboundary, IoS platforms have to integrate services from multiple domains to fulfill the requirements. Considering there are too many available services in IoS, a big challenge is how to look for a tradeoff between the construction efficiency and the precision of final solutions. For this challenge, we introduce a framework and a platform for transboundary user requirement oriented solution design in IoS. The main idea is to make use of domain priori knowledge derived from the commonness and similarities among massive historical URs and among historical integrated service solutions(ISSs). Priori knowledge is classified into three types: requirement patterns (RPs), service patterns (SPs), and probabilistic matching matrix (PMM) between RPs and SPs. A UR is modeled in the form of an intention tree (ITree) along with a set of constraints on intention nodes, and then optimal RPs are selected to cover the I-Tree as much as possible. By taking advantage of the PMM, a set of SPs are filtered out and composed together to form the final ISS. Finally, the design of a platform supporting the above process is introduced.

cs.SE↗

arXiv2020

Heterogeneous Graph Neural Network for Recommendation

The prosperous development of e-commerce has spawned diverse recommendation systems. As a matter of fact, there exist rich and complex interactions among various types of nodes in real-world recommendation systems, which can be constructed as heterogeneous graphs. How learn representative node embedding is the basis and core of the personalized recommendation system. Meta-path is a widely used structure to capture the semantics beneath such interactions and show potential ability in improving node embedding. In this paper, we propose Heterogeneous Graph neural network for Recommendation (HGRec) which injects high-order semantic into node embedding via aggregating multi-hops meta-path based neighbors and fuses rich semantics via multiple meta-paths based on attention mechanism to get comprehensive node embedding. Experimental results demonstrate the importance of rich high-order semantics and also show the potentially good interpretability of HGRec.

cs.SI↗

arXiv2020

New highly-anisotropic Rh-based Heusler compound for magnetic recording

The development of high-density magnetic recording media is limited by the superparamagnetism in very small ferromagnetic crystals. Hard magnetic materials with strong perpendicular anisotropy offer stability and high recording density. To overcome the difficulty of writing media with a large coercivity, heat assisted magnetic recording (HAMR) has been developed, rapidly heating the media to the Curie temperature Tc before writing, followed by rapid cooling. Requirements are a suitable Tc, coupled with anisotropic thermal conductivity and hard magnetic properties. Here we introduce Rh2CoSb as a new hard magnet with potential for thin film magnetic recording. A magnetocrystalline anisotropy of 3.6 MJm-3 is combined with a saturation magnetization of μ0Ms = 0.52 T at 2 K (2.2 MJm-3 and 0.44 T at room-temperature). The magnetic hardness parameter of 3.7 at room temperature is the highest observed for any rare-earth free hard magnet. The anisotropy is related to an unquenched orbital moment of 0.42 μB on Co, which is hybridized with neighbouring Rh atoms with a large spin-orbit interaction. Moreover, the pronounced temperature-dependence of the anisotropy that follows from its Tc of 450 K, together with a high thermal conductivity of 20 Wm-1K-1, makes Rh2CoSb a candidate for development for heat assisted writing with a recording density in excess of 10 Tb/in2.

cond-mat.mtrl-sci↗