arXiv subjects

Yuan Yao

Publications and source records attributed to Yuan Yao.

At least 253 records · Page 14Linked to original sources

arXiv2021

Early DSE and Automatic Generation of Coarse Grained Merged Accelerators

Post-Moore's law area-constrained systems rely on accelerators to deliver performance enhancements. Coarse grained accelerators can offer substantial domain acceleration, but manual, ad-hoc identification of code to accelerate is prohibitively expensive. Because cycle-accurate simulators and high-level synthesis flows are so time-consuming, manual creation of high-utilization accelerators that exploit control and data flow patterns at optimal granularities is rarely successful. To address these challenges, we present AccelMerger, the first automated methodology to create coarse grained, control- and data-flow-rich, merged accelerators. AccelMerger uses sequence alignment matching to recognize similar function call-graphs and loops, and neural networks to quickly evaluate their post-HLS characteristics. It accurately identifies which functions to accelerate, and it merges accelerators to respect an area budget and to accommodate system communication characteristics like latency and bandwidth. Merging two accelerators can save as much as 99% of the area of one. The space saved is used by a globally optimal integer linear program to allocate more accelerators for increased performance. We demonstate AccelMerger's effectiveness using HLS flows without any manual effort to fine-tune the resulting designs. On FPGA-based systems, AccelMerger yields application performance improvements of up to 16.7x over software implementations, and 1.91x on average with respect to state-of-the-art early-stage design space exploration tools.

cs.AR↗

arXiv2021

Accurate energies of transition metal atoms, ions, and monoxides using selected configuration interaction and density-based basis-set corrections

The semistochastic heat-bath configuration interaction (SHCI) method is a selected configuration interaction plus perturbation theory method that has provided near-full configuration interaction (FCI) levels of accuracy for many systems with both single- and multi-reference character. However, obtaining accurate energies in the complete basis set limit is hindered by the slow convergence of the FCI energy with respect to basis size. Here we show that the recently developed basis-set correction method based on range-separated density-functional theory can be used to significantly speed up basis-set convergence in SHCI calculations. In particular, we study two such schemes that differ in the functional used, and apply them to transition metal atoms and monoxides to obtain total, ionization, and dissociation energies well converged to the complete-basis-set limit within chemical accuracy.

physics.chem-ph↗

arXiv2021

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks

Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of natural language processing, some attack methods have been proposed and achieve very high attack success rates on multiple popular models. Nevertheless, there are few studies on defending against textual backdoor attacks. In this paper, we propose a simple and effective textual backdoor defense named ONION, which is based on outlier word detection and, to the best of our knowledge, is the first method that can handle all the textual backdoor attack situations. Experiments demonstrate the effectiveness of our model in defending BiLSTM and BERT against five different backdoor attacks. All the code and data of this paper can be obtained at https://github.com/thunlp/ONION.

cs.CL↗

arXiv2021

The dynamic chirality flips of Skyrmion bubbles

Magnetic skyrmion, a topological magnetic domain with complex non-coplanar spin texture, appears a disk-like structure in two dimensions. Exploring three-dimensional spin texture and related chirality switching has drawn enormous interests from the perspective of fundamental research. Here, the three-dimensional magnetic moment of the skyrmion bubbles in centrosymmetric Mn-Ni-Ga were reconstructed with the vector field tomography approach via Lorentz transmission electron microscopy. The type of the bubbles was determined from investigating the magnetic vectors in entire space. We found that the bubbles switched their chirality easily but still keep the polarity to remain the singularity of the bubbles within the material. Our results offer valuable insights into the fundamental mechanisms underlying the spin chirality flips dynamics of skyrmion bubbles.

cond-mat.mtrl-sci↗

arXiv2021

Natural Gradient Optimization for Optical Quantum Circuits

Optical quantum circuits can be optimized using gradient descent methods, as the gates in a circuit can be parametrized by continuous parameters. However, the parameter space as seen by the cost function is not Euclidean, which means that the Euclidean gradient does not generally point in the direction of steepest ascent. In order to retrieve the steepest ascent direction, in this work we implement Natural Gradient descent in the optical quantum circuit setting, which takes the local metric tensor into account. In particular, we adapt the Natural Gradient approach to a complex-valued parameter space. We then compare the Natural Gradient approach to vanilla gradient descent and to Adam over two state preparation tasks: a single-photon source and a Gottesman-Kitaev-Preskill state source. We observe that the NG approach has a faster convergence (due in part to the possibility of using larger learning rates) and a significantly smoother decay of the cost function throughout the optimization.

quant-ph↗

arXiv2021

Federated Deep Learning with Bayesian Privacy

Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users. For Federated Learning of Deep Neural Network with billions of model parameters, existing privacy-preserving solutions are unsatisfactory. Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads rendering it almost useless in practice . Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity. However, DP is vulnerable to aggressive Bayesian restoration attacks as disclosed in the literature and demonstrated in experimental results of this work. To address the aforementioned perplexity, we propose a novel Bayesian Privacy (BP) framework which enables Bayesian restoration attacks to be formulated as the probability of reconstructing private data from observed public information. Specifically, the proposed BP framework accurately quantifies privacy loss by Kullback-Leibler (KL) Divergence between the prior distribution about the privacy data and the posterior distribution of restoration private data conditioning on exposed information}. To our best knowledge, this Bayesian Privacy analysis is the first to provides theoretical justification of secure privacy-preserving capabilities against Bayesian restoration attacks. As a concrete use case, we demonstrate that a novel federated deep learning method using private passport layers is able to simultaneously achieve high model performance, privacy-preserving capability and low computational complexity. Theoretical analysis is in accordance with empirical measurements of information leakage extensively experimented with a variety of DNN networks on image classification MNIST, CIFAR10, and CIFAR100 datasets.

cs.LG↗

arXiv2021

On ADMM in Deep Learning: Convergence and Saturation-Avoidance

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called \textit{sigmoid-ADMM pair}), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order ${\cal O}(1/k)$. Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.

cs.LG↗

arXiv2021

Multi-source Heterogeneous Domain Adaptation with Conditional Weighting Adversarial Network

Heterogeneous domain adaptation (HDA) tackles the learning of cross-domain samples with both different probability distributions and feature representations. Most of the existing HDA studies focus on the single-source scenario. In reality, however, it is not uncommon to obtain samples from multiple heterogeneous domains. In this article, we study the multisource HDA problem and propose a conditional weighting adversarial network (CWAN) to address it. The proposed CWAN adversarially learns a feature transformer, a label classifier, and a domain discriminator. To quantify the importance of different source domains, CWAN introduces a sophisticated conditional weighting scheme to calculate the weights of the source domains according to the conditional distribution divergence between the source and target domains. Different from existing weighting schemes, the proposed conditional weighting scheme not only weights the source domains but also implicitly aligns the conditional distributions during the optimization process. Experimental results clearly demonstrate that the proposed CWAN performs much better than several state-of-the-art methods on four real-world datasets.

cs.LG↗

arXiv2021

Differentially Private Federated Learning with Laplacian Smoothing

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. However, an adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models. In this paper, we investigate a utility enhancement scheme based on Laplacian smoothing for differentially private federated learning (DP-Fed-LS), where the parameter aggregation with injected Gaussian noise is improved in statistical precision without losing privacy budget. Our key observation is that the aggregated gradients in federated learning often enjoy a type of smoothness, i.e. sparsity in the graph Fourier basis with polynomial decays of Fourier coefficients as frequency grows, which can be exploited by the Laplacian smoothing efficiently. Under a prescribed differential privacy budget, convergence error bounds with tight rates are provided for DP-Fed-LS with uniform subsampling of heterogeneous Non-IID data, revealing possible utility improvement of Laplacian smoothing in effective dimensionality and variance reduction, among others. Experiments over MNIST, SVHN, and Shakespeare datasets show that the proposed method can improve model accuracy with DP-guarantee and membership privacy under both uniform and Poisson subsampling mechanisms.

cs.LG↗

arXiv2021

Coherent spin rotation-induced zero thermal expansion in MnCoSi-based spiral magnets

Materials exhibiting zero thermal expansion (ZTE), namely, volume invariance during temperature change, can resist thermal shock and are highly desired in modern industries as high-precision components. However, pure ZTE materials are rare, especially those that are metallic. Here, we report the discovery of a pure metallic ZTE material: an orthorhombic Mn1-xNixCoSi spiral magnet. The introduction of Ni can efficiently enhance the ferromagnetic exchange interaction and construct the transition from a spiral magnetic state to a ferromagnetic-like state in MnCoSi-based alloys. Systematic in situ neutron powder diffraction revealed a new cycloidal spiral magnetic structure in bc plane at ground state which would transform to the helical spiral in the ab plane with increasing temperature. Combined with Lorentz transmission electron microscopy techniques, the cycloidal and helical spin order coherently rotated at varying periods along the c axis during the magnetic transition. This spin rotation drove the continuous movement of the coupled crystalline lattice and induced a large negative thermal expansion along the a axis, eventually leading to a wide-temperature ZTE effect. Our work not only introduces a new ZTE alloy but also presents a new mechanism by which to discover or design ZTE magnets.

cond-mat.mtrl-sci↗

arXiv2021

Dimming Down LED: An Open-source Threshold Implementation on Light Encryption Device (LED) Block Cipher

Lightweight block ciphers have been widely used in applications such as RFID tags, IoTs, and network sensors. Among them, with comparable parameters, the Light Encryption Device (LED) block cipher achieves the smallest area. However, implementation of encryption algorithms manifest side-channel leakage, therefore, it is crucial to protect their design against side-channel analyses. In this paper, we present a threshold implementation of the LED cipher which has 64-bit data input and 128-bit key. The presented design splits secret information among multiple shares to achieve a higher security level. We demonstrate that our implementation can protect against first-order power side-channel attacks. As a cost, the design area is almost doubled and the maximum operating frequency is degraded by 30%. To make our design verifiable, we have also open-sourced our design online.

cs.CR↗

arXiv2021

Visual Distant Supervision for Scene Graph Generation

Scene graph generation aims to identify objects and their relations in images, providing structured image representations that can facilitate numerous applications in computer vision. However, scene graph models usually require supervised learning on large quantities of labeled data with intensive human annotation. In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data. The intuition is that by aligning commonsense knowledge bases and images, we can automatically create large-scale labeled data to provide distant supervision for visual relation learning. To alleviate the noise in distantly labeled data, we further propose a framework that iteratively estimates the probabilistic relation labels and eliminates the noisy ones. Comprehensive experimental results show that our distantly supervised model outperforms strong weakly supervised and semi-supervised baselines. By further incorporating human-labeled data in a semi-supervised fashion, our model outperforms state-of-the-art fully supervised models by a large margin (e.g., 8.3 micro- and 7.8 macro-recall@50 improvements for predicate classification in Visual Genome evaluation). We make the data and code for this paper publicly available at https://github.com/thunlp/VisualDS.

cs.CV↗

arXiv2021

Pre-Trained Models: Past, Present and Future

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophisticated pre-training objectives and huge model parameters, large-scale PTMs can effectively capture knowledge from massive labeled and unlabeled data. By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks, which has been extensively demonstrated via experimental verification and empirical analysis. It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch. In this paper, we take a deep look into the history of pre-training, especially its special relation with transfer learning and self-supervised learning, to reveal the crucial position of PTMs in the AI development spectrum. Further, we comprehensively review the latest breakthroughs of PTMs. These breakthroughs are driven by the surge of computational power and the increasing availability of data, towards four important directions: designing effective architectures, utilizing rich contexts, improving computational efficiency, and conducting interpretation and theoretical analysis. Finally, we discuss a series of open problems and research directions of PTMs, and hope our view can inspire and advance the future study of PTMs.

cs.AI↗

arXiv2021

Magnetic anisotropy and critical behavior of the quaternary van der Waals ferromagnetic material $\bf CrGe_δSi_{1-δ}Te_3$

Recently, two-dimensional ferromagnetism in the family of Chromium compounds $\rm CrXTe_3 (X=Si, Ge)$ has attracted a broad research interest. Despite the structural similarity in $\rm CrTe_6$ octahedra, the size effect of inserted Ge or Si dimer contributes to significant differences in magnetism. Here, we report a new quaternary van der Waals ferromagnetic material $\rm CrGe_δSi_{1-δ}Te_3$ synthesized by flux method. Ge substitution in Si site results in the lattice expansion, further increasing the Curie temperature and reducing the magnetic anisotropy. The critical behavior of $\rm Cr_{0.96}Ge_{0.17}Si_{0.82}Te_3$ has been studied by specific heat as well as magnetization measurements. And the extracted critical exponents are self-consistent and well-obeying the scaling laws, which are closer to the 2D Ising model with interaction decaying as $J(r)\approx r^{-3.44}$.

physics.app-ph↗

arXiv2021

A Note on Learning Rare Events in Molecular Dynamics using LSTM and Transformer

Recurrent neural networks for language models like long short-term memory (LSTM) have been utilized as a tool for modeling and predicting long term dynamics of complex stochastic molecular systems. Recently successful examples on learning slow dynamics by LSTM are given with simulation data of low dimensional reaction coordinate. However, in this report we show that the following three key factors significantly affect the performance of language model learning, namely dimensionality of reaction coordinates, temporal resolution and state partition. When applying recurrent neural networks to molecular dynamics simulation trajectories of high dimensionality, we find that rare events corresponding to the slow dynamics might be obscured by other faster dynamics of the system, and cannot be efficiently learned. Under such conditions, we find that coarse graining the conformational space into metastable states and removing recrossing events when estimating transition probabilities between states could greatly help improve the accuracy of slow dynamics learning in molecular dynamics. Moreover, we also explore other models like Transformer, which do not show superior performance than LSTM in overcoming these issues. Therefore, to learn rare events of slow molecular dynamics by LSTM and Transformer, it is critical to choose proper temporal resolution (i.e., saving intervals of MD simulation trajectories) and state partition in high resolution data, since deep neural network models might not automatically disentangle slow dynamics from fast dynamics when both are present in data influencing each other.

cs.AI↗

arXiv2021

Real-time Detection and Adaptive Mitigation of Power-based Side-Channel Leakage in SoC

Power-based side-channel is a serious security threat to the System on Chip (SoC). The secret information is leaked from the power profile of the system while a cryptographic algorithm is running. The mitigation requires efforts from both the software level and hardware level. Currently, there is no comprehensive solution that can guarantee the whole complex system is free of leakage and can generically protect all cryptographic algorithms. In this paper, we propose a real-time leakage detection and mitigation system which enables the system to monitor the side-channel leakage effects of the hardware. Our proposed system has extensions that provide a real-time monitor of power consumption, detection of side-channel leakage, and real-time adaptive mitigation of detected side-channel leakage. Our proposed system is generic and can protect any algorithm running on it.

cs.CR↗

arXiv2021

Orbital Optimization in Selected Configuration Interaction Methods

We study several approaches to orbital optimization in selected configuration interaction plus perturbation theory (SCI+PT) methods, and test them on the ground and excited states of three molecules using the semistochastic heatbath configuration interaction (SHCI) method. We discuss the ways in which the orbital optimization problem in SCI resembles and differs from that in complete active space self-consistent field (CASSCF). Starting from natural orbitals, these approaches divide into three classes of optimization methods according to how they treat coupling between configuration interaction (CI) coefficients and orbital parameters, namely uncoupled, fully coupled, and quasi-fully coupled methods. We demonstrate that taking the coupling into account is crucial for fast convergence and recommend two quasi-fully coupled methods for such applications: accelerated diagonal Newton and Broyden-Fletcher-Goldfarb-Shanno (BFGS).

physics.chem-ph↗

arXiv2021

Programmable RO (PRO): A Multipurpose Countermeasure against Side-channel and Fault Injection Attack

Side-channel and fault injection attacks reveal secret information by monitoring or manipulating the physical effects of computations involving secret variables. Circuit-level countermeasures help to deter these attacks, and traditionally such countermeasures have been developed for each attack vector separately. We demonstrate a multipurpose ring oscillator design - Programmable Ring Oscillator (PRO) to address both fault attacks and side-channel attacks in a generic, application-independent manner. PRO, as an integrated primitive, can provide on-chip side-channel resistance, power monitoring, and fault detection capabilities to a secure design. We present a grid of PROs monitoring the on-chip power network to detect anomalies. Such power anomalies may be caused by external factors such as electromagnetic fault injection and power glitches, as well as by internal factors such as hardware Trojans. By monitoring the frequency of the ring oscillators, we are able to detect the on-chip power anomaly in time as well as in location. Moreover, we show that the PROs can also inject a random noise pattern into a design's power consumption. By randomly switching the frequency of a ring oscillator, the resulting power-noise pattern significantly reduces the power-based side-channel leakage of a cipher. We discuss the design of PRO and present measurement results on a Xilinx Spartan-6 FPGA prototype, and we show that side-channel and fault vulnerabilities can be addressed at a low cost by introducing PRO to the design. We conclude that PRO can serve as an application-independent, multipurpose countermeasure.

cs.CR↗