arXiv subjects

Ying Zhang

Publications and source records attributed to Ying Zhang.

At least 523 records · Page 29Linked to original sources

arXiv2017

Recurrent Neural Networks With Limited Numerical Precision

Recurrent Neural Networks (RNNs) produce state-of-art performance on many machine learning tasks but their demand on resources in terms of memory and computational power are often high. Therefore, there is a great interest in optimizing the computations performed with these models especially when considering development of specialized low-power hardware for deep networks. One way of reducing the computational needs is to limit the numerical precision of the network weights and biases, and this will be addressed for the case of RNNs. We present results from the use of different stochastic and deterministic reduced precision training methods applied to two major RNN types, which are then tested on three datasets. The results show that the stochastic and deterministic ternarization, pow2- ternarization, and exponential quantization methods gave rise to low-precision RNNs that produce similar and even higher accuracy on certain datasets, therefore providing a path towards training more efficient implementations of RNNs in specialized hardware.

cs.NE↗

arXiv2017

Influence of pairing correlations on the radius of neutron-rich nuclei

The influence of pairing correlations on the neutron root mean square (rms) radius of nuclei is investigated in the framework of self-consistent Skyrme Hartree-Fock-Bogoliubov calculations. The continuum is treated appropriately by the Green's function techniques. As an example the nucleus $^{124}$Zr is treated for a varying strength of pairing correlations. We find that, as the pairing strength increases, the neutron rms radius first shrinks, reaches a minimum and beyond this point it expands again. The shrinkage is due to the the so-called `pairing anti-halo effect', i. e. due to the decreasing of the asymptotic density distribution with increasing pairing. However, in some cases, increasing pairing correlations can also lead to an expansion of the nucleus due to a growing occupation of so-called `halo' orbits, i.e. weakly bound states and resonances in the continuum with low-$\ell $ values. In this case, the neutron radii are extended just by the influence of pairing correlations, since these `halo' orbits cannot be occupied without pairing. The term `anti-halo effect' is not justified in such cases. For a full understanding of this complicated interplay self-consistent calculations are necessary.

nucl-th↗

arXiv2017

Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the state-of-the-art in various benchmarks. Meanwhile, Connectionist Temporal Classification (CTC) with Recurrent Neural Networks (RNNs), which is proposed for labeling unsegmented sequences, makes it feasible to train an end-to-end speech recognition system instead of hybrid settings. However, RNNs are computationally expensive and sometimes difficult to train. In this paper, inspired by the advantages of both CNNs and the CTC approach, we propose an end-to-end speech framework for sequence labeling, by combining hierarchical CNNs with CTC directly without recurrent connections. By evaluating the approach on the TIMIT phoneme recognition task, we show that the proposed model is not only computationally efficient, but also competitive with the existing baseline systems. Moreover, we argue that CNNs have the capability to model temporal correlations with appropriate context information.

cs.CL↗

arXiv2016

End-to-End Data Visualization by Metric Learning and Coordinate Transformation

This paper presents a deep nonlinear metric learning framework for data visualization on an image dataset. We propose the Triangular Similarity and prove its equivalence to the Cosine Similarity in measuring a data pair. Based on this novel similarity, a geometrically motivated loss function - the triangular loss - is then developed for optimizing a metric learning system comprising two identical CNNs. It is shown that this deep nonlinear system can be efficiently trained by a hybrid algorithm based on the conventional backpropagation algorithm. More interestingly, benefiting from classical manifold learning theories, the proposed system offers two different views to visualize the outputs, the second of which provides better classification results than the state-of-the-art methods in the visualizable spaces.

cs.CV↗

arXiv2016

Nuclear matter properties with nucleon-nucleon forces up to fifth order in the chiral expansion

The properties of nuclear matter are studied using state-of-the-art nucleon-nucleon forces up to fifth order in chiral effective field theory. The equations of state of symmetric nuclear matter and pure neutron matter are calculated in the framework of the Brueckner-Hartree-Fock theory. We discuss in detail the convergence pattern of the chiral expansion and the regulator dependence of the calculated equations of state and provide an estimation of the truncation uncertainty. For all employed values of the regulator, the fifth-order chiral two-nucleon potential is found to generate nuclear saturation properties similar to the available phenomenological high precision potentials. We also extract the symmetry energy of nuclear matter, which is shown to be quite robust with respect to the chiral order and the value of the regulator.

nucl-th↗

arXiv2016

Fitting MA(q) Models in the Closed Invertible Region

The use of reparameterization in the maximization of the likelihood function of the MA(q) model is discussed. A general method for testing for the presence of a parameter estimate on the boundary of an MA(q) model is presented. This test is illustrated with a brief simulation experiment for the MA(q) for q=1,2,3,4 in which it is shown that the probability of an estimate being on the boundary increases with q.

math.ST↗

arXiv2016

On Multiplicative Integration with Recurrent Neural Networks

We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the computational building block of an RNN, while introducing almost no extra parameters. The new structure can be easily embedded into many popular RNN models, including LSTMs and GRUs. We empirically analyze its learning behaviour and conduct evaluations on several tasks using different RNN models. Our experimental results demonstrate that Multiplicative Integration can provide a substantial performance boost over many of the existing RNN models.

cs.LG↗

arXiv2016

Top-k Spatial-keyword Publish/Subscribe Over Sliding Window

With the prevalence of social media and GPS-enabled devices, a massive amount of geo-textual data has been generated in a stream fashion, leading to a variety of applications such as location-based recommendation and information dissemination. In this paper, we investigate a novel real-time top-k monitoring problem over sliding window of streaming data; that is, we continuously maintain the top-k most relevant geo-textual messages (e.g., geo-tagged tweets) for a large number of spatial-keyword subscriptions (e.g., registered users interested in local events) simultaneously. To provide the most recent information under controllable memory cost, sliding window model is employed on the streaming geo-textual data. To the best of our knowledge, this is the first work to study top-k spatial-keyword publish/subscribe over sliding window. A novel centralized system, called Skype (Topk Spatial-keyword Publish/Subscribe), is proposed in this paper. In Skype, to continuously maintain top-k results for massive subscriptions, we devise a novel indexing structure upon subscriptions such that each incoming message can be immediately delivered on its arrival. To reduce the expensive top-k re-evaluation cost triggered by message expiration, we develop a novel cost-based k-skyband technique to reduce the number of re-evaluations in a cost-effective way. Extensive experiments verify the great efficiency and effectiveness of our proposed techniques. Furthermore, to support better scalability and higher throughput, we propose a distributed version of Skype, namely, DSkype, on top of Storm, which is a popular distributed stream processing system. With the help of fine-tuned subscription/message distribution mechanisms, DSkype can achieve orders of magnitude speed-up than its centralized version.

cs.DB↗

arXiv2016

When Engagement Meets Similarity: Efficient (k,r)-Core Computation on Social Networks

In this paper, we investigate the problem of (k,r)-core which intends to find cohesive subgraphs on social networks considering both user engagement and similarity perspectives. In particular, we adopt the popular concept of k-core to guarantee the engagement of the users (vertices) in a group (subgraph) where each vertex in a (k,r)-core connects to at least k other vertices. Meanwhile, we also consider the pairwise similarity between users based on their profiles. For a given similarity metric and a similarity threshold r, the similarity between any two vertices in a (k,r)-core is ensured not less than r. Efficient algorithms are proposed to enumerate all maximal (k,r)-cores and find the maximum (k,r)-core, where both problems are shown to be NP-hard. Effective pruning techniques significantly reduce the search space of two algorithms and a novel (k,k')-core based (k,r)-core size upper bound enhances performance of the maximum (k,r)-core computation. We also devise effective search orders to accommodate the different nature of two mining algorithms. Comprehensive experiments on real-life data demonstrate that the maximal/maximum (k,r)-cores enable us to find interesting cohesive subgraphs, and performance of two mining algorithms is significantly improved by proposed techniques.

cs.SI↗

arXiv2016

Computer Algebra Derivation of the Bias of Burg Estimators

A symbolic method is discussed which can be used to obtain the asymptotic bias and variance to order $O(1/n)$ for estimators in stationary time series. Using this method the bias to $O(1/n)$ of the Burg estimator in AR(1) and AR(2) models is shown to be equal to that of the least squares estimators in both the known and unknown mean cases. Previous researchers have only been able to obtain simulation results for this bias because this problem is too intractable without using computer algebra.

math.ST↗

arXiv2016

Partial autocorrelation parameterization for subset autoregression

A new version of the partial autocorrelation plot and a new family of subset autoregressive models are introduced. A comprehensive approach to model identification, estimation and diagnostic checking is developed for these models. These models are better suited to efficient model building of high-order autoregressions with long time series. Several illustrative examples are given.

math.ST↗

arXiv2016

Faster ARMA maximum likelihood estimation

A new likelihood based AR approximation is given for ARMA models. The usual algorithms for the computation of the likelihood of an ARMA model require $O(n)$ flops per function evaluation. Using our new approximation, an algorithm is developed which requires only $O(1)$ flops in repeated likelihood evaluations. In most cases, the new algorithm gives results identical to or very close to the exact maximum likelihood estimate (MLE). This algorithm is easily implemented in high level Quantitative Programming Environments (QPEs) such as {\it Mathematica\/}, MatLab and R. In order to obtain reasonable speed, previous ARMA maximum likelihood algorithms are usually implemented in C or some other machine efficient language. With our algorithm it is easy to do maximum likelihood estimation for long time series directly in the QPE of your choice. The new algorithm is extended to obtain the MLE for the mean parameter. Simulation experiments which illustrate the effectiveness of the new algorithm are discussed. {\it Mathematica\/} and R packages which implement the algorithm discussed in this paper are available (McLeod and Zhang, 2007). Based on these package implementations, it is expected that the interested researcher would be able to implement this algorithm in other QPE's.

math.ST↗

arXiv2016

Developments in Maximum Likelihood Unit Root Tests

The exact maximum likelihood estimate (MLE) provides a test statistic for the unit root test that is more powerful \citep[p. 577]{Fuller96} than the usual least squares approach. In this paper a new derivation is given for the asymptotic distribution of this test statistic that is simpler and more direct than the previous method. The response surface regression method is used to obtain a fast algorithm that computes accurate finite-sample critical values. This algorithm is available in the R package {\tt mleur} that is available on CRAN. The empirical power of the new test is shown to be much better than the usual test not only in the normal case but also for innovations generated from an infinite variance stable distribution as well as for innovations generated from a GARCH$(1,1)$ process.

math.ST↗

arXiv2016

Professor Forcing: A New Algorithm for Training Recurrent Networks

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network's own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

stat.ML↗

arXiv2016

A Fast Order-Based Approach for Core Maintenance

Graphs have been widely used in many applications such as social networks, collaboration networks, and biological networks. One important graph analytics is to explore cohesive subgraphs in a large graph. Among several cohesive subgraphs studied, k-core is one that can be computed in linear time for a static graph. Since graphs are evolving in real applications, in this paper, we study core maintenance which is to reduce the computational cost to compute k-cores for a graph when graphs are updated from time to time dynamically. We identify drawbacks of the existing efficient algorithm, which needs a large search space to find the vertices that need to be updated, and has high overhead to maintain the index built, when a graph is updated. We propose a new order-based approach to maintain an order, called k-order, among vertices, while a graph is updated. Our new algorithm can significantly outperform the state-of-the-art algorithm up to 3 orders of magnitude for the 11 large real graphs tested. We report our findings in this paper.

cs.DB↗

arXiv2016

Approximate Nearest Neighbor Search on High Dimensional Data --- Experiments, Analyses, and Improvement (v1.0)

Approximate Nearest neighbor search (ANNS) is fundamental and essential operation in applications from many domains, such as databases, machine learning, multimedia, and computer vision. Although many algorithms have been continuously proposed in the literature in the above domains each year, there is no comprehensive evaluation and analysis of their performances. In this paper, we conduct a comprehensive experimental evaluation of many state-of-the-art methods for approximate nearest neighbor search. Our study (1) is cross-disciplinary (i.e., including 16 algorithms in different domains, and from practitioners) and (2) has evaluated a diverse range of settings, including 20 datasets, several evaluation metrics, and different query workloads. The experimental results are carefully reported and analyzed to understand the performance results. Furthermore, we propose a new method that achieves both high query efficiency and high recall empirically on majority of the datasets under a wide range of settings.

cs.DB↗

arXiv2016

Stringer: Balancing Latency and Resource Usage in Service Function Chain Provisioning

Network Functions Virtualization, or NFV, enables telecommunications infrastructure providers to replace special-purpose networking equipment with commodity servers running virtualized network functions (VNFs). A service provider utilizing NFV technology faces the SFC provisioning problem of assigning VNF instances to nodes in the physical infrastructure (e.g., a datacenter), and routing Service Function Chains (sequences of functions required by customers, a.k.a. SFCs) in the physical network. In doing so, the provider must balance between various competing goals of performance and resource usage. We present an approach for SFC provisioning, consisting of three elements. The first element is a fast, scalable round-robin heuristic. The second element is a Mixed Integer Programming (MIP) based approach. The third element is a queueing-theoretic model to estimate the average latency associated with any SFC provisioning solution. Combined, these elements create an approach that generates a set of SFC provisioning solutions, reflecting different tradeoffs between resource usage and performance.

cs.NI↗

arXiv2016

A centrosymmetric hexagonal magnet with superstable biskyrmion magnetic nanodomains in a wide temperature range of 100K to 340K

Here, we report on the experimental discovery of biskyrmion magnetic nanodomains at RT and the observation of a biskyrmion-derived topological Hall effect (THE) in the centrosymmetric hexagonal MnNiGa magnet. Using a phase reconstruction technique based on a transport-of-intensity equation (TIE), we established the texture of the biskyrmion spin. Results from Lorentz transmission electron microscopy (TEM) and the topological Hall effect revealed that the biskyrmion phase is stable over a much wider temperature range (100 K to ~340K) and a larger magnetic field range in our material than in skyrmion-hosting bulk crystals reported previously[1-3,6]. The super-wide operating temperature and the broad range of material options indicate important progress toward the realization of skyrmion-based spintronic devices.

cond-mat.mtrl-sci↗