arXiv subjects
Publications and source records attributed to Yuan Zhang.
We prove that the critical percolation parameter for Finitary Random Interlacements (FRI) is continuous with respect to the path length parameter $T$. The proof uses a result which is interesting on its own right; equality of natural critical parameters for FRI percolation phase transition.
Channel pruning is one of the major compression approaches for deep neural networks. While previous pruning methods have mostly focused on identifying unimportant channels, channel pruning is considered as a special case of neural architecture search in recent years. However, existing methods are either complicated or prone to sub-optimal pruning. In this paper, we propose a pruning framework that adaptively determines the number of each layer's channels as well as the wights inheritance criteria for sub-network. Firstly, evaluate the importance of each block in the network based on the mean of the scaling parameters of the BN layers. Secondly, use the bisection method to quickly find the compact sub-network satisfying the budget. Finally, adaptively and efficiently choose the weight inheritance criterion that fits the current architecture and fine-tune the pruned network to recover performance. AdaPruner allows to obtain pruned network quickly, accurately and efficiently, taking into account both the structure and initialization weights. We prune the currently popular CNN models (VGG, ResNet, MobileNetV2) on different image classification datasets, and the experimental results demonstrate the effectiveness of our proposed method. On ImageNet, we reduce 32.8% FLOPs of MobileNetV2 with only 0.62% decrease for top-1 accuracy, which exceeds all previous state-of-the-art channel pruning methods. The code will be released.
Graph Attention Network (GAT) focuses on modelling simple undirected and single relational graph data only. This limits its ability to deal with more general and complex multi-relational graphs that contain entities with directed links of different labels (e.g., knowledge graphs). Therefore, directly applying GAT on multi-relational graphs leads to sub-optimal solutions. To tackle this issue, we propose r-GAT, a relational graph attention network to learn multi-channel entity representations. Specifically, each channel corresponds to a latent semantic aspect of an entity. This enables us to aggregate neighborhood information for the current aspect using relation features. We further propose a query-aware attention mechanism for subsequent tasks to select useful aspects. Extensive experiments on link prediction and entity classification tasks show that our r-GAT can model multi-relational graphs effectively. Also, we show the interpretability of our approach by case study.
Nowadays, scene text recognition has attracted more and more attention due to its various applications. Most state-of-the-art methods adopt an encoder-decoder framework with attention mechanism, which generates text autoregressively from left to right. Despite the convincing performance, the speed is limited because of the one-by-one decoding strategy. As opposed to autoregressive models, non-autoregressive models predict the results in parallel with a much shorter inference time, but the accuracy falls behind the autoregressive counterpart considerably. In this paper, we propose a Parallel, Iterative and Mimicking Network (PIMNet) to balance accuracy and efficiency. Specifically, PIMNet adopts a parallel attention mechanism to predict the text faster and an iterative generation mechanism to make the predictions more accurate. In each iteration, the context information is fully explored. To improve learning of the hidden layer, we exploit the mimicking learning in the training phase, where an additional autoregressive decoder is adopted and the parallel decoder mimics the autoregressive decoder with fitting outputs of the hidden layer. With the shared backbone between the two decoders, the proposed PIMNet can be trained end-to-end without pre-training. During inference, the branch of the autoregressive decoder is removed for a faster speed. Extensive experiments on public benchmarks demonstrate the effectiveness and efficiency of PIMNet. Our code will be available at https://github.com/Pay20Y/PIMNet.
This paper investigates several cost-sparsity induced optimal input selection problems for structured systems. Given are an autonomous system and a prescribed set of input links, where each input link has a non-negative cost. The problems include selecting the minimum cost of input links, and selecting the input links with the smallest possible cost with a bound on their cardinality, all to ensure system structural controllability. Current studies show that in the dedicated input case (i.e., each input can actuate only a state variable), the former problem is polynomially solvable by some graph-theoretic algorithms, while the general nontrivial constrained case is largely unexploited. We show these problems can be formulated as equivalent integer linear programming (ILP) problems. Subject to a certain condition on the prescribed input configurations that contains the dedicated input one as a special case, we demonstrate that the constraint matrices of these ILPs are totally unimodular. This property allows us to solve those ILPs efficiently simply via their linear programming (LP) relaxations, leading to a unifying algebraic method for these problems with polynomial time complexity. It is further shown those problems could be solved in strongly polynomial time, independent of the size of the costs and cardinality bounds. Finally, an example is provided to illustrate the power of the proposed method.
Knowledge distillation is a technique to enhance the generalization ability of a student model by exploiting outputs from a teacher model. Recently, feature-map based variants explore knowledge transfer between manually assigned teacher-student pairs in intermediate layers for further improvement. However, layer semantics may vary in different neural networks and semantic mismatch in manual layer associations will lead to performance degeneration due to negative regularization. To address this issue, we propose Semantic Calibration for cross-layer Knowledge Distillation (SemCKD), which automatically assigns proper target layers of the teacher model for each student layer with an attention mechanism. With a learned attention distribution, each student layer distills knowledge contained in multiple teacher layers rather than a specific intermediate layer for appropriate cross-layer supervision. We further provide theoretical analysis of the association weights and conduct extensive experiments to demonstrate the effectiveness of our approach. Code is avaliable at \url{https://github.com/DefangChen/SemCKD}.
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with spoken user queries. In this paper, we propose a novel architecture that jointly solves the NER and EL tasks by combining them in a joint reranking module. We show that our proposed framework improves NER accuracy by up to 3.13% and EL accuracy by up to 3.6% in F1 score. The features used also lead to better accuracies in other natural language understanding tasks, such as domain classification and semantic parsing.
Spiking neural network (SNN), compared with depth neural network (DNN), has faster processing speed, lower energy consumption and more biological interpretability, which is expected to approach Strong AI. Reinforcement learning is similar to learning in biology. It is of great significance to study the combination of SNN and RL. We propose the reinforcement learning method of spike distillation network (SDN) with STBP. This method uses distillation to effectively avoid the weakness of STBP, which can achieve SOTA performance in classification, and can obtain a smaller, faster convergence and lower power consumption SNN reinforcement learning model. Experiments show that our method can converge faster than traditional SNN reinforcement learning and DNN reinforcement learning methods, about 1000 epochs faster, and obtain SNN 200 times smaller than DNN. We also deploy SDN to the PKU nc64c chip, which proves that SDN has lower power consumption than DNN, and the power consumption of SDN is more than 600 times lower than DNN on large-scale devices. SDN provides a new way of SNN reinforcement learning, and can achieve SOTA performance, which proves the possibility of further development of SNN reinforcement learning.
We introduce a unified framework, formulated as general latent space models, to study complex higher-order network interactions among multiple entities. Our framework covers several popular models in recent network analysis literature, including mixture multi-layer latent space model and hypergraph latent space model. We formulate the relationship between the latent positions and the observed data via a generalized multilinear kernel as the link function. While our model enjoys decent generality, its maximum likelihood parameter estimation is also convenient via a generalized tensor decomposition procedure.We propose a novel algorithm using projected gradient descent on Grassmannians. We also develop original theoretical guarantees for our algorithm. First, we show its linear convergence under mild conditions. Second, we establish finite-sample statistical error rates of latent position estimation, determined by the signal strength, degrees of freedom and the smoothness of link function, for both general and specific latent space models. We demonstrate the effectiveness of our method on synthetic data. We also showcase the merit of our method on two real-world datasets that are conventionally described by different specific models in producing meaningful and interpretable parameter estimations and accurate link prediction. We demonstrate the effectiveness of our method on synthetic data. We also showcase the merit of our method on two real-world datasets that are conventionally described by different specific models in producing meaningful and interpretable parameter estimations and accurate link prediction.
Perovskite solar cell (PSC) is one of the most promising next generation photovoltaic technologies, and there are considerable interests in the role of possible polarization of organic-inorganic halide perovskites (OIHPs) in photovoltaic conversion. The polarity of OIHPs, however, is still hotly debated. In this review, we examine recent literature on the polarity of OIHPs from both theoretical and experimental points of view, and argue that they can be both polar and nonpolar, depending on compositions, processing, and environments. Implications of OIHP polarity to photovoltaic conversion is also discussed, and effort in answering these questions continues to render us new insights. In the future, integrating local scanning probe with global macroscopic measurements in-situ will provide invaluable microscopic insight into the intriguing macroscopic phenomena, while synchrotron diffractions and scanning transmission electron microscopy on more stable samples may ultimately settle the debate.
Recent experiments have demonstrated Rabi-oscillations, superradiant pulses and stimulated emission from negatively-charged nitrogen-vacancy ($\mathrm{NV}^{-}$) center spins in microwave resonators. These phenomena witness the kind of collective and strong coupling which has been prerequisite for observation of superradiant lasing in the optical frequency regime. In this article, we investigate the possibility to employ coherence, present in both the collective $\mathrm{NV}^{-}$ spin ensemble and the microwave field, to achieve a superradiant maser. Our calculations show that a superradiant maser with a linewidth below millihertz can be achieved with moderate kilohertz incoherent pumping of over $10^{14}$ spins kept at low temperature. We show that the superradiant masing prevails in the presence of inhomogeneous broadening, and we present numerical and analytical studies of the dependence of the phenomenon on the various physical parameters.
In recent years, video data has dominated internet traffic and becomes one of the major data formats. With the emerging 5G and internet of things (IoT) technologies, more and more videos are generated by edge devices, sent across networks, and consumed by machines. The volume of video consumed by machine is exceeding the volume of video consumed by humans. Machine vision tasks include object detection, segmentation, tracking, and other machine-based applications, which are quite different from those for human consumption. On the other hand, due to large volumes of video data, it is essential to compress video before transmission. Thus, efficient video coding for machines (VCM) has become an important topic in academia and industry. In July 2019, the international standardization organization, i.e., MPEG, created an Ad-Hoc group named VCM to study the requirements for potential standardization work. In this paper, we will address the recent development activities in the MPEG VCM group. Specifically, we will first provide an overview of the MPEG VCM group including use cases, requirements, processing pipelines, plan for potential VCM standards, followed by the evaluation framework including machine-vision tasks, dataset, evaluation metrics, and anchor generation. We then introduce technology solutions proposed so far and discuss the recent responses to the Call for Evidence issued by MPEG VCM group.
We develop a stochastic mean-field theory to describe active frequency measurements of pulsed superradiant emission, studied in recent experiments with strontium-87 atoms trapped in an optical lattice inside an optical cavity [M. Norcia, et al., Phys. Rev. X 8, 21036 (2018)]. Our theory reveals the intriguing dynamics of atomic ensembles with multiple transition frequencies, and it reproduces the superradiant beats signal, noisy power spectra, and frequency uncertainty in remarkable agreement with the experiments. Moreover, by reducing the number of atoms, elongating the superradiant pulses and shortening the experimental duty cycle, we predict a short-term frequency uncertainty $9\times10^{-16} \sqrt{τ/s}$, which makes active frequency measurements with superradiant transitions comparable with the record performance of current frequency standards [M. Schioppo, et al., Nat. Photonics, 11, 48 (2017)]. Our theory combines cavity-quantum electrodynamics and quantum measurement theory, and it can be readily applied to explore conditional quantum dynamics and describe frequency measurements for other processes such as steady-state superradiance and superradiant Raman lasing.
Let $a$ and $b$ be relatively prime positive integers. In this paper the weighted sum $\sum_{n\in{\rm NR}(a,b)}λ^{n-1}n^m$ is given explicitly or in terms of the Apostol-Bernoulli numbers, where $m$ is a nonnegative integer, and ${\rm NR}(a,b)$ denotes the set of positive integers nonrepresentable in terms of $a$ and $b$.
This paper proposes a novel notion for structural controllability under structured numerical perturbations, namely the perturbation-tolerant structural controllability (PTSC), on a single-input structured system whose entries can be classified into three categories: fixed zero entries, unknown generic entries whose values are fixed but unknown, and perturbed entries that can take arbitrary complex values. Such a system is PTSC if, for almost all values of the unknown generic entries in the parameter space, the corresponding controllable system realizations can preserve controllability under arbitrary complex-valued perturbations with their structure prescribed by the perturbed entries. This new notion can characterize the generic property in controllability preservation under structured numerical perturbations. We give a necessary and sufficient condition for a single-input system to be PTSC, whose verification has polynomial time complexity. Our results can serve as some feasibility conditions for the conventional structured controllability radius problems from a generic view.
Topological phases of matter give rise to exotic physics that can be leveraged for next generation quantum computation and spintronic devices. Thus, the search for topological phases and the quantum states that they exhibit have become the subject of a massive research effort in condensed matter physics. Topologically protected states have been produced in a variety of systems, including artificial lattices, graphene nanoribbons (GNRs) and bismuth bilayers. Despite these advances, the real-time manipulation of individual topological states and their relative coupling, a necessary feature for the realization of topological qubits, remains elusive. Guided by first-principles calculations, we spatially manipulate robust, zero-dimensional topological states by altering the topological invariants of quasi-one-dimensional artificial graphene nanostructures. This is achieved by positioning carbon monoxide molecules on a copper surface to confine its surface state electrons into artificial atoms positioned to emulate the low-energy electronic structure of graphene derivatives. Ultimately, we demonstrate control over the coupling between adjacent topological states that are finely engineered and simulate complex Hamiltonians. Our atomic synthesis gives access to an infinite range of nanoribbon geometries, including those beyond the current reach of synthetic chemistry, and thus provides an ideal platform for the design and study of novel topological and quantum states of matter.
Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization. While specialized model architectures and pre-training of seq2seq models have been proposed to address this issue, the former often comes at the cost of generality and the latter only shows limited success. In this paper, we study the impact of intermediate representations on compositional generalization in pre-trained seq2seq models, without changing the model architecture at all, and identify key aspects for designing effective representations. Instead of training to directly map natural language to an executable form, we map to a reversible or lossy intermediate representation that has stronger structural correspondence with natural language. The combination of our proposed intermediate representations and pre-trained models is surprisingly effective, where the best combinations obtain a new state-of-the-art on CFQ (+14.8 accuracy points) and on the template-splits of three text-to-SQL datasets (+15.0 to +19.4 accuracy points). This work highlights that intermediate representations provide an important and potentially overlooked degree of freedom for improving the compositional generalization abilities of pre-trained seq2seq models.
Speech enhancement is challenging because of the diversity of background noise types. Most of the existing methods are focused on modelling the speech rather than the noise. In this paper, we propose a novel idea to model speech and noise simultaneously in a two-branch convolutional neural network, namely SN-Net. In SN-Net, the two branches predict speech and noise, respectively. Instead of information fusion only at the final output layer, interaction modules are introduced at several intermediate feature domains between the two branches to benefit each other. Such an interaction can leverage features learned from one branch to counteract the undesired part and restore the missing component of the other and thus enhance their discrimination capabilities. We also design a feature extraction module, namely residual-convolution-and-attention (RA), to capture the correlations along temporal and frequency dimensions for both the speech and the noises. Evaluations on public datasets show that the interaction module plays a key role in simultaneous modeling and the SN-Net outperforms the state-of-the-art by a large margin on various evaluation metrics. The proposed SN-Net also shows superior performance for speaker separation.