arXiv subjects

Shuai Zhang

Publications and source records attributed to Shuai Zhang.

At least 361 records · Page 20Linked to original sources

arXiv2020

Suspicious Massive Registration Detection via Dynamic Heterogeneous Graph Neural Networks

Massive account registration has raised concerns on risk management in e-commerce companies, especially when registration increases rapidly within a short time frame. To monitor these registrations constantly and minimize the potential loss they might incur, detecting massive registration and predicting their riskiness are necessary. In this paper, we propose a Dynamic Heterogeneous Graph Neural Network framework to capture suspicious massive registrations (DHGReg). We first construct a dynamic heterogeneous graph from the registration data, which is composed of a structural subgraph and a temporal subgraph. Then, we design an efficient architecture to predict suspicious/benign accounts. Our proposed model outperforms the baseline models and is computationally efficient in processing a dynamic heterogeneous graph constructed from a real-world dataset. In practice, the DHGReg framework would benefit the detection of suspicious registration behaviors at an early stage.

cs.LG↗

arXiv2020

Fast and Efficient Parallel Breadth-First Search with Power-law Graph Transformation

In the big data era, graph computing is widely used to exploit the hidden value in real-world graphs in various scenarios such as social networks, knowledge graphs, web searching, and recommendation systems. However, the random memory accesses result in inefficient use of cache and the irregular degree distribution leads to substantial load imbalance. Breadth-First Search (BFS) is frequently utilized as a kernel for many important and complex graph algorithms. In this paper, we describe a preprocessing approach using Reverse Cuthill-Mckee (RCM) algorithm to improve data locality and demonstrate how to achieve an efficient load balancing for BFS. Computations on RCM-reordered graph data are also accelerated with SIMD executions. We evaluate the performance of the graph preprocessing approach on Kronecker graphs of the Graph500 benchmark and real-world graphs. Our BFS implementation on RCM-reordered graph data achieves 326.48 MTEPS/W (mega TEPS per watt) on an ARMv8 system, ranking 2nd on the Green Graph500 list in June 2020 (the 1st rank uses GPU acceleration).

cs.DS↗

arXiv2020

First-Principles Equation of State Database for Warm Dense Matter Computation

We put together a first-principles equation of state (FPEOS) database for matter at extreme conditions by combining results from path integral Monte Carlo and density functional molecular dynamics simulations of the elements H, He, B, C, N, O, Ne, Na, Mg, Al and Si as well as the compounds LiF, B4C, BN, CH4, CH2, C2H3, CH, C2H, MgO, and MgSiO3. For all these materials, we provide the pressure and internal energy over a density-temperature range from ~0.5 to 50 g/cc and from ~10^4 to 10^9 K, which are based on ~5000 different first-principles simulations. We compute isobars, adiabats and shock Hugoniot curves in the regime of L and K shell ionization. Invoking the linear mixing approximation, we study the properties of mixtures at high density and temperature. We derive the Hugoniot curves for water and alumina as well as for carbon-oxygen, helium-neon, and CH-silicon mixtures. We predict the maximal shock compression ratios of H2O, H2O2, Al2O3, CO, and CO2 to be 4.61, 4.64, 4.64, 4.89, and 4.83, respectively. Finally we use the FPEOS database to determine the points of maximum shock compression for all available binary mixtures. We identify mixtures that reach higher shock compression ratios than their endmembers. We discuss trends common to all mixtures in pressure-temperature and particle-shock velocity spaces. In the supplementary material, we provide all FPEOS tables as well as computer codes for interpolation, Hugoniot calculations, and plots of various thermodynamic functions.

cond-mat.mtrl-sci↗

arXiv2020

Learning User Representations with Hypercuboids for Recommender Systems

Modeling user interests is crucial in real-world recommender systems. In this paper, we present a new user interest representation model for personalized recommendation. Specifically, the key novelty behind our model is that it explicitly models user interests as a hypercuboid instead of a point in the space. In our approach, the recommendation score is learned by calculating a compositional distance between the user hypercuboid and the item. This helps to alleviate the potential geometric inflexibility of existing collaborative filtering approaches, enabling a greater extent of modeling capability. Furthermore, we present two variants of hypercuboids to enhance the capability in capturing the diversities of user interests. A neural architecture is also proposed to facilitate user hypercuboid learning by capturing the activity sequences (e.g., buy and rate) of users. We demonstrate the effectiveness of our proposed model via extensive experiments on both public and commercial datasets. Empirical results show that our approach achieves very promising results, outperforming existing state-of-the-art.

cs.IR↗

arXiv2020

Decoupling Pronunciation and Language for End-to-end Code-switching Automatic Speech Recognition

Despite the recent significant advances witnessed in end-to-end (E2E) ASR system for code-switching, hunger for audio-text paired data limits the further improvement of the models' performance. In this paper, we propose a decoupled transformer model to use monolingual paired data and unpaired text data to alleviate the problem of code-switching data shortage. The model is decoupled into two parts: audio-to-phoneme (A2P) network and phoneme-to-text (P2T) network. The A2P network can learn acoustic pattern scenarios using large-scale monolingual paired data. Meanwhile, it generates multiple phoneme sequence candidates for single audio data in real-time during the training process. Then the generated phoneme-text paired data is used to train the P2T network. This network can be pre-trained with large amounts of external unpaired text data. By using monolingual data and unpaired text data, the decoupled transformer model reduces the high dependency on code-switching paired training data of E2E model to a certain extent. Finally, the two networks are optimized jointly through attention fusion. We evaluate the proposed method on the public Mandarin-English code-switching dataset. Compared with our transformer baseline, the proposed method achieves 18.14% relative mix error rate reduction.

cs.SD↗

arXiv2020

Nonideal Mixing Effects in Warm Dense Matter Studied with First-Principles Computer Simulations

We study nonideal mixing effects in the regime of warm dense matter (WDM) by computing the shock Hugoniot curves of BN, MgO, and MgSiO_3. First, we derive these curves from the equations of state (EOS) of the fully interacting systems, which were obtained using a combination of path integral Monte Carlo calculations at high temperature and density functional molecular dynamics simulations at lower temperatures. We then use the ideal mixing approximation at constant pressure and temperature to rederive these Hugoniot curves from the EOS tables of the individual elements. We find that the linear mixing approximation works remarkably well at temperatures above ~2*10^5 K, where the shock compression ratio exceeds ~3.2. The shape of the Hugoniot curve of each compound is well reproduced. Regions of increased shock compression, that emerge because of the ionization of L and K shell electrons, are well represented and the maximum compression ratio on the Hugoniot curves is reproduced with high precision. Some deviations are seen near the onset of the L shell ionization regime, where ionization equilibrium in the fully interacting system cannot be well reproduced by the ideal mixing approximation. This approximation also breaks down at lower temperatures, where chemical bonds play an increasingly import role. However, the results imply that equilibrium properties of binary and ternary mixtures in the regime of WDM can be derived from the EOS tables of the individual elements. This significantly simplifies the characterization of binary and ternary mixtures in the WDM and plasma phases, which otherwise requires large numbers of more computationally expensive first-principles computer simulations.

cond-mat.mtrl-sci↗

arXiv2020

Performance Optimization for Federated Person Re-identification via Benchmark Analysis

Federated learning is a privacy-preserving machine learning technique that learns a shared model across decentralized clients. It can alleviate privacy concerns of personal re-identification, an important computer vision task. In this work, we implement federated learning to person re-identification (FedReID) and optimize its performance affected by statistical heterogeneity in the real-world scenario. We first construct a new benchmark to investigate the performance of FedReID. This benchmark consists of (1) nine datasets with different volumes sourced from different domains to simulate the heterogeneous situation in reality, (2) two federated scenarios, and (3) an enhanced federated algorithm for FedReID. The benchmark analysis shows that the client-edge-cloud architecture, represented by the federated-by-dataset scenario, has better performance than client-server architecture in FedReID. It also reveals the bottlenecks of FedReID under the real-world scenario, including poor performance of large datasets caused by unbalanced weights in model aggregation and challenges in convergence. Then we propose two optimization methods: (1) To address the unbalanced weight problem, we propose a new method to dynamically change the weights according to the scale of model changes in clients in each training round; (2) To facilitate convergence, we adopt knowledge distillation to refine the server model with knowledge generated from client models on a public dataset. Experiment results demonstrate that our strategies can achieve much better convergence with superior performance on all datasets. We believe that our work will inspire the community to further explore the implementation of federated learning on more computer vision tasks in real-world scenarios.

cs.CV↗

arXiv2020

Reversible Engineering of Topological Insulator Surface State Conductivity through Optical Excitation

Despite the broadband response, limited optical absorption at a particular wavelength hinders the development of optoelectronics based on Dirac fermions. Heterostructures of graphene and various semiconductors have been explored for this purpose, while non-ideal interfaces often limit the performance. The topological insulator is a natural hybrid system, with the surface states hosting high-mobility Dirac fermions and the small-bandgap semiconducting bulk state strongly absorbing light. In this work, we show a large photocurrent response from a field effect transistor device based on intrinsic topological insulator Sn-Bi1.1Sb0.9Te2S. The photocurrent response is non-volatile and sensitively depends on the initial Fermi energy of the surface state, and it can be erased by controlling the gate voltage. Our observations can be explained with a remote photo-doping mechanism, in which the light excites the defects in the bulk and frees the localized carriers to the surface state. This photodoping modulates the surface state conductivity without compromising the mobility, and it also significantly modify the quantum Hall effect of the surface state. Our work thus illustrates a route to reversibly manipulate the surface states through optical excitation, shedding light into utilizing topological surface states for quantum optoelectronics.

cond-mat.mes-hall↗

arXiv2020

A Practical Chinese Dependency Parser Based on A Large-scale Dataset

Dependency parsing is a longstanding natural language processing task, with its outputs crucial to various downstream tasks. Recently, neural network based (NN-based) dependency parsing has achieved significant progress and obtained the state-of-the-art results. As we all know, NN-based approaches require massive amounts of labeled training data, which is very expensive because it requires human annotation by experts. Thus few industrial-oriented dependency parser tools are publicly available. In this report, we present Baidu Dependency Parser (DDParser), a new Chinese dependency parser trained on a large-scale manually labeled dataset called Baidu Chinese Treebank (DuCTB). DuCTB consists of about one million annotated sentences from multiple sources including search logs, Chinese newswire, various forum discourses, and conversation programs. DDParser is extended on the graph-based biaffine parser to accommodate to the characteristics of Chinese dataset. We conduct experiments on two test sets: the standard test set with the same distribution as the training set and the random test set sampled from other sources, and the labeled attachment scores (LAS) of them are 92.9% and 86.9% respectively. DDParser achieves the state-of-the-art results, and is released at https://github.com/baidu/DDParser.

cs.CL↗

arXiv2020

Computational prediction of RNA tertiary structures using machine learning methods

RNAs play crucial and versatile roles in biological processes. Computational prediction approaches can help to understand RNA structures and their stabilizing factors, thus providing information on their functions, and facilitating the design of new RNAs. Machine learning (ML) techniques have made tremendous progress in many fields in the past few years. Although their usage in protein-related fields has a long history, the use of ML methods in predicting RNA tertiary structures is new and rare. Here, we review the recent advances of using ML methods on RNA structure predictions and discuss the advantages and limitation, the difficulties and potentials of these approaches when applied in the field.

physics.bio-ph↗

arXiv2020

TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling

Transformer has been widely-used in many Natural Language Processing (NLP) tasks and the scaled dot-product attention between tokens is a core module of Transformer. This attention is a token-wise design and its complexity is quadratic to the length of sequence, limiting its application potential for long sequence tasks. In this paper, we propose a dimension-wise attention mechanism based on which a novel language modeling approach (namely TensorCoder) can be developed. The dimension-wise attention can reduce the attention complexity from the original $O(N^2d)$ to $O(Nd^2)$, where $N$ is the length of the sequence and $d$ is the dimensionality of head. We verify TensorCoder on two tasks including masked language modeling and neural machine translation. Compared with the original Transformer, TensorCoder not only greatly reduces the calculation of the original model but also obtains improved performance on masked language modeling task (in PTB dataset) and comparable performance on machine translation tasks.

cs.CL↗

arXiv2020

Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition

Although attention based end-to-end models have achieved promising performance in speech recognition, the multi-pass forward computation in beam-search increases inference time cost, which limits their practical applications. To address this issue, we propose a non-autoregressive end-to-end speech recognition system called LASO (listen attentively, and spell once). Because of the non-autoregressive property, LASO predicts a textual token in the sequence without the dependence on other tokens. Without beam-search, the one-pass propagation much reduces inference time cost of LASO. And because the model is based on the attention based feedforward structure, the computation can be implemented in parallel efficiently. We conduct experiments on publicly available Chinese dataset AISHELL-1. LASO achieves a character error rate of 6.4%, which outperforms the state-of-the-art autoregressive transformer model (6.7%). The average inference latency is 21 ms, which is 1/50 of the autoregressive transformer model.

eess.AS↗

arXiv2020

Benchmarking boron carbide equation of state using computation and experiment

Boron carbide (B$_4$C) is of both fundamental scientific and practical interest in inertial confinement fusion (ICF) and high energy density physics experiments. We report the results of a comprehensive computational study of the equation of state (EOS) of B$_4$C in the liquid, warm dense matter, and plasma phases. Our calculations are cross-validated by comparisons with Hugoniot measurements up to 61 megabar from planar shock experiments performed at the National Ignition Facility (NIF). Our computational methods include path integral Monte Carlo, activity expansion, as well as all-electron Green's function Korringa-Kohn-Rostoker and molecular dynamics that are both based on density functional theory. We calculate the pressure-internal energy EOS of B$_4$C over a broad range of temperatures ($\sim$6$\times$10$^3$--5$\times$10$^8$ K) and densities (0.025--50 g/cm$^{3}$). We assess that the largest discrepancies between theoretical predictions are $\lesssim$5% near the compression maximum at 1--2$\times10^6$ K. This is the warm-dense state in which the K shell significantly ionizes and has posed grand challenges to theory and experiment. By comparing with different EOS models, we find a Purgatorio model (LEOS 2122) that agrees with our calculations. The maximum discrepancies in pressure between our first-principles predictions and LEOS 2122 are $\sim$18% and occur at temperatures between 6$\times$10$^3$--2$\times$10$^5$ K, which we believe originate from differences in the ion thermal term and the cold curve that are modeled in LEOS 2122 in comparison with our first-principles calculations. In addition, we have developed three new equation of state models and applied them to 1D hydrodynamic simulations of a polar direct-drive NIF implosion, demonstrating that these new models are now available for future ICF design studies.

physics.plasm-ph↗

arXiv2020

DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition

Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem. In this work, we first characterize this phenomenon from the information-theoretic perspective and show that under certain conditions, the mutual information between the output after $l$ layers and the input of GCN converges to 0 exponentially with respect to $l$. We also show that, on the other hand, graph decomposition can potentially weaken the condition of such convergence rate, which enabled our analysis for GraphCNN. While different graph structures can only benefit from the corresponding decomposition, in practice, we propose an automatic connectivity-aware graph decomposition algorithm, DeGNN, to improve the performance of general graph neural networks. Extensive experiments on widely adopted benchmark datasets demonstrate that DeGNN can not only significantly boost the performance of corresponding GNNs, but also achieves the state-of-the-art performances.

cs.LG↗

arXiv2020

Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case

Although graph neural networks (GNNs) have made great progress recently on learning from graph-structured data in practice, their theoretical guarantee on generalizability remains elusive in the literature. In this paper, we provide a theoretically-grounded generalizability analysis of GNNs with one hidden layer for both regression and binary classification problems. Under the assumption that there exists a ground-truth GNN model (with zero generalization error), the objective of GNN learning is to estimate the ground-truth GNN parameters from the training data. To achieve this objective, we propose a learning algorithm that is built on tensor initialization and accelerated gradient descent. We then show that the proposed learning algorithm converges to the ground-truth GNN model for the regression problem, and to a model sufficiently close to the ground-truth for the binary classification problem. Moreover, for both cases, the convergence rate of the proposed learning algorithm is proven to be linear and faster than the vanilla gradient descent algorithm. We further explore the relationship between the sample complexity of GNNs and their underlying graph properties. Lastly, we provide numerical experiments to demonstrate the validity of our analysis and the effectiveness of the proposed learning algorithm for GNNs.

cs.LG↗

arXiv2020

Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning

The chest X-Ray (CXR) is the one of the most common clinical exam used to diagnose thoracic diseases and abnormalities. The volume of CXR scans generated daily in hospitals is huge. Therefore, an automated diagnosis system able to save the effort of doctors is of great value. At present, the applications of artificial intelligence in CXR diagnosis usually use pattern recognition to classify the scans. However, such methods rely on labeled databases, which are costly and usually have large error rates. In this work, we built a database containing more than 12,000 CXR scans and radiological reports, and developed a model based on deep convolutional neural network and recurrent network with attention mechanism. The model learns features from the CXR scans and the associated raw radiological reports directly; no additional labeling of the scans are needed. The model provides automated recognition of given scans and generation of reports. The quality of the generated reports was evaluated with both the CIDEr scores and by radiologists as well. The CIDEr scores are found to be around 5.8 on average for the testing dataset. Further blind evaluation suggested a comparable performance against human radiologist.

cs.CV↗

arXiv2020

Quantifying the dynamics of protein self-organization using deep learning analysis of atomic force microscopy data

Dynamics of protein self-assembly on the inorganic surface and the resultant geometric patterns are visualized using high-speed atomic force microscopy. The time dynamics of the classical macroscopic descriptors such as 2D Fast Fourier Transforms (FFT), correlation and pair distribution function are explored using the unsupervised linear unmixing, demonstrating the presence of static ordered and dynamic disordered phases and establishing their time dynamics. The deep learning (DL)-based workflow is developed to analyze detailed particle dynamics on the particle-by-particle level. Beyond the macroscopic descriptors, we utilize the knowledge of local particle geometries and configurations to explore the evolution of local geometries and reconstruct the interaction potential between the particles. Finally, we use the machine learning-based feature extraction to define particle neighborhood free of physics constraints. This approach allowed separating the possible classes of particle behavior, identify the associated transition probabilities, and further extend this analysis to identify slow modes and associated configurations, allowing for systematic exploration and predictive modeling of the time dynamics of the system. Overall, this work establishes the DL based workflow for the analysis of the self-organization processes in complex systems from observational data and provides insight into the fundamental mechanisms.

physics.comp-ph↗

arXiv2020

Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition

Non-autoregressive transformer models have achieved extremely fast inference speed and comparable performance with autoregressive sequence-to-sequence models in neural machine translation. Most of the non-autoregressive transformers decode the target sequence from a predefined-length mask sequence. If the predefined length is too long, it will cause a lot of redundant calculations. If the predefined length is shorter than the length of the target sequence, it will hurt the performance of the model. To address this problem and improve the inference speed, we propose a spike-triggered non-autoregressive transformer model for end-to-end speech recognition, which introduces a CTC module to predict the length of the target sequence and accelerate the convergence. All the experiments are conducted on a public Chinese mandarin dataset AISHELL-1. The results show that the proposed model can accurately predict the length of the target sequence and achieve a competitive performance with the advanced transformers. What's more, the model even achieves a real-time factor of 0.0056, which exceeds all mainstream speech recognition models.

eess.AS↗