arXiv subjects

Jun Zhu

Publications and source records attributed to Jun Zhu.

At least 451 records · Page 25Linked to original sources

arXiv2019

Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks

Deep neural networks are vulnerable to adversarial examples, which can mislead classifiers by adding imperceptible perturbations. An intriguing property of adversarial examples is their good transferability, making black-box attacks feasible in real-world applications. Due to the threat of adversarial attacks, many methods have been proposed to improve the robustness. Several state-of-the-art defenses are shown to be robust against transferable adversarial examples. In this paper, we propose a translation-invariant attack method to generate more transferable adversarial examples against the defense models. By optimizing a perturbation over an ensemble of translated images, the generated adversarial example is less sensitive to the white-box model being attacked and has better transferability. To improve the efficiency of attacks, we further show that our method can be implemented by convolving the gradient at the untranslated image with a pre-defined kernel. Our method is generally applicable to any gradient-based attack method. Extensive experiments on the ImageNet dataset validate the effectiveness of the proposed method. Our best attack fools eight state-of-the-art defenses at an 82% success rate on average based only on the transferability, demonstrating the insecurity of the current defense techniques.

cs.CV↗

arXiv2019

Spin relaxation in fluorinated single and bilayer graphene

We present a joint experiment-theory study on the role of fluorine adatoms in spin and momentum scattering of charge carriers in dilute fluorinated graphene and bilayer graphene. The experimental spin-flip and momentum scattering rates and their dependence on the density of fluorine and carrier doping are obtained through weak localization and conductivity measurements, respectively, and suggest the role of fluorine as resonant magnetic impurities. For the estimated fluorine concentration of a few 100 ppm, the observed spin lifetimes are in the range of 1-10\,ps. Theoretically, we established tight-binding electronic structures of fluorinated graphene and bilayer graphene by fitting to density functional supercell calculations and performed a comprehensive analysis of the spin-flip and momentum scattering rates within the same devices, aiming to develop a consistent description of both scattering channels. We find that resonant scattering in graphene is very sensitive to the precise position of the resonance level, as well as to the magnitude of the exchange coupling between itinerant carriers and localized spins. The experimental data point to the presence of weak spin-flip scatterers that, at the same time, relax the electron momentum strongly, nearly preserving the electron-hole symmetry. Such scatterers would exhibit resonance energies much closer to the neutrality point than what density functional theory predicts in the dilute limit. The inclusion of a magnetic moment on fluorine adatoms allowed us to qualitatively capture the carrier density dependence of the experimental rates but predicts a greater (weaker) spin (momentum) relaxation rate than the measurements. We discuss possible scenarios that may be responsible for the discrepancies. Our systematic study exposes the complexities involved in accurately capturing the behavior of adatoms on graphene.

cond-mat.mes-hall↗

arXiv2019

Metallic Phase and Temperature Dependence of the $ν= 0$ Quantum Hall State in Bilayer Graphene

The $ν= 0$ quantum Hall state of bilayer graphene is a fertile playground to realize many-body ground states with various broken symmetries. Here we report the experimental observations of a previously unreported metallic phase. The metallic phase resides in the phase space between the previously identified layer polarized state at large transverse electric field and the canted antiferromagnetic state at small transverse electric field. We also report temperature dependence studies of the quantum spin Hall state of $ν= 0$. Complex nonmonotonic behavior reveals concomitant bulk and edge conductions and excitations. These results provide a timely experimental update to understand the rich physics of the $ν= 0$ state.

cond-mat.mes-hall↗

arXiv2019

Secure Massive MIMO Communication with Low-resolution DACs

In this paper, we investigate secure transmission in a massive multiple-input multiple-output (MIMO) system adopting low-resolution digital-to-analog converters (DACs). Artificial noise (AN) is deliberately transmitted simultaneously with the confidential signals to degrade the eavesdropper's channel quality. By applying the Bussgang theorem, a DAC quantization model is developed which facilitates the analysis of the asymptotic achievable secrecy rate. Interestingly, for a fixed power allocation factor $ϕ$, low-resolution DACs typically result in a secrecy rate loss, but in certain cases they provide superior performance, e.g., at low signal-to-noise ratio (SNR). Specifically, we derive a closed-form SNR threshold which determines whether low-resolution or high-resolution DACs are preferable for improving the secrecy rate. Furthermore, a closed-form expression for the optimal $ϕ$ is derived. With AN generated in the null-space of the user channel and the optimal $ϕ$, low-resolution DACs inevitably cause secrecy rate loss. On the other hand, for random AN with the optimal $ϕ$, the secrecy rate is hardly affected by the DAC resolution because the negative impact of the quantization noise can be compensated for by reducing the AN power. All the derived analytical results are verified by numerical simulations.

cs.IT↗

arXiv2019

Reward Shaping via Meta-Learning

Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL). However, designing shaping functions usually requires much expert knowledge and hand-engineering, and the difficulties are further exacerbated given multiple similar tasks to solve. In this paper, we consider reward shaping on a distribution of tasks, and propose a general meta-learning framework to automatically learn the efficient reward shaping on newly sampled tasks, assuming only shared state space but not necessarily action space. We first derive the theoretically optimal reward shaping in terms of credit assignment in model-free RL. We then propose a value-based meta-learning algorithm to extract an effective prior over the optimal reward shaping. The prior can be applied directly to new tasks, or provably adapted to the task-posterior while solving the task within few gradient updates. We demonstrate the effectiveness of our shaping through significantly improved learning efficiency and interpretable visualizations across various settings, including notably a successful transfer from DQN to DDPG.

cs.LG↗

arXiv2019

Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it is imperative to develop algorithms to learn models with good interpretability (Doshi-Velez 2017). An important factor that leads to the lack of interpretability of DNNs is the ambiguity of neurons, where a neuron may fire for various unrelated concepts. This work aims to increase the interpretability of DNNs on the whole image space by reducing the ambiguity of neurons. In this paper, we make the following contributions: 1) We propose a metric to evaluate the consistency level of neurons in a network quantitatively. 2) We find that the learned features of neurons are ambiguous by leveraging adversarial examples. 3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.

cs.LG↗

arXiv2018

Lazy-CFR: fast and near optimal regret minimization for extensive games with imperfect information

Counterfactual regret minimization (CFR) is the most popular algorithm on solving two-player zero-sum extensive games with imperfect information and achieves state-of-the-art performance in practice. However, the performance of CFR is not fully understood, since empirical results on the regret are much better than the upper bound proved in \cite{zinkevich2008regret}. Another issue is that CFR has to traverse the whole game tree in each round, which is time-consuming in large scale games. In this paper, we present a novel technique, lazy update, which can avoid traversing the whole game tree in CFR, as well as a novel analysis on the regret of CFR with lazy update. Our analysis can also be applied to the vanilla CFR, resulting in a much tighter regret bound than that in \cite{zinkevich2008regret}. Inspired by lazy update, we further present a novel CFR variant, named Lazy-CFR. Compared to traversing $O(|\mathcal{I}|)$ information sets in vanilla CFR, Lazy-CFR needs only to traverse $O(\sqrt{|\mathcal{I}|})$ information sets per round while keeping the regret bound almost the same, where $\mathcal{I}$ is the class of all information sets. As a result, Lazy-CFR shows better convergence result compared with vanilla CFR. Experimental results consistently show that Lazy-CFR outperforms the vanilla CFR significantly.

cs.LG↗

arXiv2018

Graphical Generative Adversarial Networks

We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data. Graphical-GAN conjoins the power of Bayesian networks on compactly representing the dependency structures among random variables and that of generative adversarial networks on learning expressive dependency functions. We introduce a structured recognition model to infer the posterior distribution of latent variables given observations. We generalize the Expectation Propagation (EP) algorithm to learn the generative model and recognition model jointly. Finally, we present two important instances of Graphical-GAN, i.e. Gaussian Mixture GAN (GMGAN) and State Space GAN (SSGAN), which can successfully learn the discrete and temporal structures on visual datasets, respectively.

cs.LG↗

arXiv2018

A valley valve and electron beam splitter

Developing alternative paradigms of electronics beyond silicon technology requires the exploration of fundamentally new physical mechanisms, such as the valley specific phenomena in hexagonal two-dimensional materials. We realize ballistic valley Hall kink states in bilayer graphene and demonstrate gate-controlled current transmission in a four-kink router device. The operations of a waveguide, a valve and a tunable electron beam splitter are demonstrated. The valley valve exploits the valley-momentum locking of the kink states and reaches an on/off ratio of 8 at zero magnetic field. A magnetic field enables a full-ranged tunable coherent beam splitter. These results pave the path to building a scalable, coherent quantum transportation network based on the kink states.

cond-mat.mes-hall↗

arXiv2018

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention. Yet now, SNNs have not shown competitive performance compared with artificial neural networks (ANNs), due to the lack of effective learning algorithms and efficient programming frameworks. We address this issue from two aspects: (1) We propose a neuron normalization technique to adjust the neural selectivity and develop a direct learning algorithm for deep SNNs. (2) Via narrowing the rate coding window and converting the leaky integrate-and-fire (LIF) model into an explicitly iterative version, we present a Pytorch-based implementation method towards the training of large-scale SNNs. In this way, we are able to train deep SNNs with tens of times speedup. As a result, we achieve significantly better accuracy than the reported works on neuromorphic datasets (N-MNIST and DVS-CIFAR10), and comparable accuracy as existing ANNs and pre-trained SNNs on non-spiking datasets (CIFAR10). {To our best knowledge, this is the first work that demonstrates direct training of deep SNNs with high performance on CIFAR10, and the efficient implementation provides a new way to explore the potential of SNNs.

cs.NE↗

arXiv2018

MIMO Channel Information Feedback Using Deep Recurrent Network

In a multiple-input multiple-output (MIMO) system, the availability of channel state information (CSI) at the transmitter is essential for performance improvement. Recent convolutional neural network (NN) based techniques show competitive ability in realizing CSI compression and feedback. By introducing a new NN architecture, we enhance the accuracy of quantized CSI feedback in MIMO communications. The proposed NN architecture invokes a module named long short-term memory (LSTM) which admits the NN to benefit from exploiting temporal and frequency correlations of wireless channels. Compromising performance with complexity, we further modify the NN architecture with a significantly reduced number of parameters to be trained. Finally, experiments show that the proposed NN architectures achieve better performance in terms of both CSI compression and recovery accuracy.

cs.IT↗

arXiv2018

Composite Binary Decomposition Networks

Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.

cs.LG↗

arXiv2018

Towards Robust Detection of Adversarial Examples

Although the recent progress is substantial, deep learning methods can be vulnerable to the maliciously generated adversarial examples. In this paper, we present a novel training procedure and a thresholding test strategy, towards robust detection of adversarial examples. In training, we propose to minimize the reverse cross-entropy (RCE), which encourages a deep network to learn latent representations that better distinguish adversarial examples from normal ones. In testing, we propose to use a thresholding strategy as the detector to filter out adversarial examples for reliable predictions. Our method is simple to implement using standard algorithms, with little extra training cost compared to the common cross-entropy minimization. We apply our method to defend various attacking methods on the widely used MNIST and CIFAR-10 datasets, and achieve significant improvements on robust predictions under all the threat models in the adversarial setting.

cs.LG↗

arXiv2018

Semi-crowdsourced Clustering with Deep Generative Models

We consider the semi-supervised clustering problem where crowdsourcing provides noisy information about the pairwise comparisons on a small subset of data, i.e., whether a sample pair is in the same cluster. We propose a new approach that includes a deep generative model (DGM) to characterize low-level features of the data, and a statistical relational model for noisy pairwise annotations on its subset. The two parts share the latent variables. To make the model automatically trade-off between its complexity and fitting data, we also develop its fully Bayesian variant. The challenge of inference is addressed by fast (natural-gradient) stochastic variational inference algorithms, where we effectively combine variational message passing for the relational part and amortized learning of the DGM under a unified framework. Empirical results on synthetic and real-world datasets show that our model outperforms previous crowdsourced clustering methods.

stat.ML↗

arXiv2018

Analyzing the Noise Robustness of Deep Neural Networks

Deep neural networks (DNNs) are vulnerable to maliciously generated adversarial examples. These examples are intentionally designed by making imperceptible perturbations and often mislead a DNN into making an incorrect prediction. This phenomenon means that there is significant risk in applying DNNs to safety-critical applications, such as driverless cars. To address this issue, we present a visual analytics approach to explain the primary cause of the wrong predictions introduced by adversarial examples. The key is to analyze the datapaths of the adversarial examples and compare them with those of the normal examples. A datapath is a group of critical neurons and their connections. To this end, we formulate the datapath extraction as a subset selection problem and approximately solve it based on back-propagation. A multi-level visualization consisting of a segmented DAG (layer level), an Euler diagram (feature map level), and a heat map (neuron level), has been designed to help experts investigate datapaths from the high-level layers to the detailed neuron activations. Two case studies are conducted that demonstrate the promise of our approach in support of explaining the working mechanism of adversarial examples.

cs.LG↗

arXiv2018

A Hierarchical Recurrent Neural Network for Symbolic Melody Generation

In recent years, neural networks have been used to generate symbolic melodies. However, the long-term structure in the melody has posed great difficulty for designing a good model. In this paper, we present a hierarchical recurrent neural network for melody generation, which consists of three Long-Short-Term-Memory (LSTM) subnetworks working in a coarse-to-fine manner along time. Specifically, the three subnetworks generate bar profiles, beat profiles and notes in turn, and the output of the high-level subnetworks are fed into the low-level subnetworks, serving as guidance for generating the finer time-scale melody components in low-level subnetworks. Two human behavior experiments demonstrate the advantage of this structure over the single-layer LSTM which attempts to learn all hidden structures in melodies. Compared with the state-of-the-art models MidiNet and MusicVAE, the hierarchical recurrent neural network produces better melodies evaluated by humans.

cs.SD↗

arXiv2018

Deep Structured Generative Models

Deep generative models have shown promising results in generating realistic images, but it is still non-trivial to generate images with complicated structures. The main reason is that most of the current generative models fail to explore the structures in the images including spatial layout and semantic relations between objects. To address this issue, we propose a novel deep structured generative model which boosts generative adversarial networks (GANs) with the aid of structure information. In particular, the layout or structure of the scene is encoded by a stochastic and-or graph (sAOG), in which the terminal nodes represent single objects and edges represent relations between objects. With the sAOG appropriately harnessed, our model can successfully capture the intrinsic structure in the scenes and generate images of complicated scenes accordingly. Furthermore, a detection network is introduced to infer scene structures from a image. Experimental results demonstrate the effectiveness of our proposed method on both modeling the intrinsic structures, and generating realistic images.

cs.LG↗

arXiv2018

Collaborative Filtering with User-Item Co-Autoregressive Models

Deep neural networks have shown promise in collaborative filtering (CF). However, existing neural approaches are either user-based or item-based, which cannot leverage all the underlying information explicitly. We propose CF-UIcA, a neural co-autoregressive model for CF tasks, which exploits the structural correlation in the domains of both users and items. The co-autoregression allows extra desired properties to be incorporated for different tasks. Furthermore, we develop an efficient stochastic learning algorithm to handle large scale datasets. We evaluate CF-UIcA on two popular benchmarks: MovieLens 1M and Netflix, and achieve state-of-the-art performance in both rating prediction and top-N recommendation tasks, which demonstrates the effectiveness of CF-UIcA.

cs.LG↗