arXiv subjects

Jingyi Zhang

Publications and source records attributed to Jingyi Zhang.

At least 109 records · Page 6Linked to original sources

arXiv2020

Learning Source Phrase Representations for Neural Machine Translation

The Transformer translation model (Vaswani et al., 2017) based on a multi-head attention mechanism can be computed effectively in parallel and has significantly pushed forward the performance of Neural Machine Translation (NMT). Though intuitively the attentional network can connect distant words via shorter network paths than RNNs, empirical analysis demonstrates that it still has difficulty in fully capturing long-distance dependencies (Tang et al., 2018). Considering that modeling phrases instead of words has significantly improved the Statistical Machine Translation (SMT) approach through the use of larger translation blocks ("phrases") and its reordering ability, modeling NMT at phrase level is an intuitive proposal to help the model capture long-distance relationships. In this paper, we first propose an attentive phrase representation generation mechanism which is able to generate phrase representations from corresponding token representations. In addition, we incorporate the generated phrase representations into the Transformer translation model to enhance its ability to capture long-distance relationships. In our experiments, we obtain significant improvements on the WMT 14 English-German and English-French tasks on top of the strong Transformer baseline, which shows the effectiveness of our approach. Our approach helps Transformer Base models perform at the level of Transformer Big models, and even significantly better for long sentences, but with substantially fewer parameters and training steps. The fact that phrase representations help even in the big setting further supports our conjecture that they make a valuable contribution to long-distance relations.

cs.CL↗

arXiv2020

Lipschitz Constrained Parameter Initialization for Deep Transformers

The Transformer translation model employs residual connection and layer normalization to ease the optimization difficulties caused by its multi-layer encoder/decoder structure. Previous research shows that even with residual connection and layer normalization, deep Transformers still have difficulty in training, and particularly Transformer models with more than 12 encoder/decoder layers fail to converge. In this paper, we first empirically demonstrate that a simple modification made in the official implementation, which changes the computation order of residual connection and layer normalization, can significantly ease the optimization of deep Transformers. We then compare the subtle differences in computation order in considerable detail, and present a parameter initialization method that leverages the Lipschitz constraint on the initialization of Transformer parameters that effectively ensures training convergence. In contrast to findings in previous research we further demonstrate that with Lipschitz parameter initialization, deep Transformers with the original computation order can converge, and obtain significant BLEU improvements with up to 24 layers. In contrast to previous research which focuses on deep encoders, our approach additionally enables Transformers to also benefit from deep decoders.

cs.CL↗

arXiv2020

More efficient approximation of smoothing splines via space-filling basis selection

We consider the problem of approximating smoothing spline estimators in a nonparametric regression model. When applied to a sample of size $n$, the smoothing spline estimator can be expressed as a linear combination of $n$ basis functions, requiring $O(n^3)$ computational time when the number of predictors $d\geq 2$. Such a sizable computational cost hinders the broad applicability of smoothing splines. In practice, the full sample smoothing spline estimator can be approximated by an estimator based on $q$ randomly-selected basis functions, resulting in a computational cost of $O(nq^2)$. It is known that these two estimators converge at the identical rate when $q$ is of the order $O\{n^{2/(pr+1)}\}$, where $p\in [1,2]$ depends on the true function $η$, and $r > 1$ depends on the type of spline. Such $q$ is called the essential number of basis functions. In this article, we develop a more efficient basis selection method. By selecting the ones corresponding to roughly equal-spaced observations, the proposed method chooses a set of basis functions with a large diversity. The asymptotic analysis shows our proposed smoothing spline estimator can decrease $q$ to roughly $O\{n^{1/(pr+1)}\}$, when $d\leq pr+1$. Applications on synthetic and real-world datasets show the proposed method leads to a smaller prediction error compared with other basis selection methods.

stat.CO↗

arXiv2019

A New Strategy for Estimating Photometric Redshifts of Quasars

Based on the SDSS and SDSS-WISE quasar datasets, we put forward two schemes to estimate the photometric redshifts of quasars. Our schemes are based on the idea that the samples are firstly classified into subsamples by a classifier and then photometric redshift estimation of different subsamples is performed by a regressor. Random Forest is adopted as the core algorithm of the classifiers, while Random Forest and kNN are applied as the key algorithms of regressors. The samples are divided into two subsamples and four subsamples depending on the redshift distribution. The performance based on different samples, different algorithms and different schemes are compared. The experimental results indicate that the accuracy of photometric redshift estimation for the two schemes generally improve to some extent compared to the original scheme in terms of the percents in \frac{|Δz|}{1+z_{i}}<0.1 and \frac{|Δz|}{1+z_{i}}<0.2 and mean absolute error. Only given the running speed, kNN shows its superiority to Random Forest. The performance of Random Forest is a little better than or comparable to that of kNN with the two datasets. The accuracy based on the SDSS-WISE sample outperforms that based on the SDSS sample no matter by kNN or by Random Forest. More information from more bands is considered and helpful to improve the accuracy of photometric redshift estimation. Evidently it can be found that our strategy to estimate photometric redshift is applicable and may be applied to other datasets or other kinds of objects. Only talking about the percent in \frac{|Δz|}{1+z_{i}}<0.3, there is still large room for further improvement in the photometric redshift estimation.

astro-ph.IM↗

arXiv2019

Efficient Selection of Quasar Candidates Based on Optical and Infrared Photometric Data Using Machine Learning

We aim to select quasar candidates based on the two large survey databases, Pan-STARRS and AllWISE. Exploring the distribution of quasars and stars in the color spaces, we find that the combination of infrared and optical photometry is more conducive to select quasar candidates. Two new color criterions (yW1W2 and izW1W2) are constructed to distinguish quasars from stars efficiently. With izW1W2, 98.30% of star contamination is eliminated, while 99.50% of quasars are retained, at least to the magnitude limit of our training set of stars. Based on the optical and infrared color features, we put forward an efficient schema to select quasar candidates and high redshift quasar candidates, in which two machine learning algorithms (XGBoost and SVM) are implemented. The XGBoost and SVM classifiers have proven to be very effective with accuracy of 99.46% when 8Color as input pattern and default model parameters. Applying the two optimal classifiers to the unknown Pan-STARRS and AllWISE cross-matched data set, a total of 2,006,632 intersected sources are predicted to be quasar candidates given quasar probability larger than 0.5 (i.e. P_QSO>0.5). Among them, 1,201,211 have high probability (P_QSO>0.95). For these newly predicted quasar candidates, a regressor is constructed to estimate their redshifts. Finally 7,402 z>3.5 quasars are obtained. Given the magnitude limitation and site of the LAMOST telescope, part of these candidates will be used as the input catalogue of the LAMOST telescope for follow-up observation, and the rest may be observed by other telescopes.

astro-ph.IM↗

arXiv2018

Guiding Neural Machine Translation with Retrieved Translation Pieces

One of the difficulties of neural machine translation (NMT) is the recall and appropriate translation of low-frequency words or phrases. In this paper, we propose a simple, fast, and effective method for recalling previously seen translation examples and incorporating them into the NMT decoding process. Specifically, for an input sentence, we use a search engine to retrieve sentence pairs whose source sides are similar with the input sentence, and then collect $n$-grams that are both in the retrieved target sentences and aligned with words that match in the source sentences, which we call "translation pieces". We compute pseudo-probabilities for each retrieved sentence based on similarities between the input sentence and the retrieved source sentences, and use these to weight the retrieved translation pieces. Finally, an existing NMT model is used to translate the input sentence, with an additional bonus given to outputs that contain the collected translation pieces. We show our method improves NMT translation results up to 6 BLEU points on three narrow domain translation tasks where repetitiveness of the target sentences is particularly salient. It also causes little increase in the translation time, and compares favorably to another alternative retrieval-based method with respect to accuracy, speed, and simplicity of implementation.

cs.CL↗

arXiv2018

The global monopole spacetime and its topological charge

In this paper, we show that the global monopole spacetime is one of the exact solutions of Einstein equations by means of the method treating the matter field as a non-linear sigma model, without the weak field approximation applied in the original derivation by Barriola and Vilenkin in 1989. Furthermore, we find the physical origin of the topological charge in the global monopole spacetime. Finally, we generalize the proposal which generates spacetime from thermodynamical laws to the case that spacetime with global monopole charge.

gr-qc↗

arXiv2018

Imbalanced learning for RR Lyrae stars

We apply machine learning and Convex-Hull algorithms to separate RR Lyrae stars from other stars, like main sequence stars, white dwarf stars, carbon stars, CVs and carbon-lines stars, based on the Sloan Digital Sky Survey (SDSS) and Galaxy Evolution Explorer (GALEX). In the low-dimensional space, the Convex-Hull algorithm is applied to select RR Lyrae stars. Given different input patterns of (u-g, g-r), (g-r, r-i), (r-i, i-z), (u-g, g-r, r-i), (g-r, r-i, i-z), (u-g, g-r, i-z) and (u-g, r-i, i-z), different convex hulls can be built for RR Lyrae stars. Comparing the performance of different input patterns, u-g, g-r, i-z is the best input pattern. For this input pattern, the efficiency (the fraction of true RR Lyrae stars in the predicted RR Lyrae sample) is 4.2% with a completeness (the fraction of recovered RR Lyrae stars in the whole RR Lyrae sample) of 100%, increases to 9.9% with 97% completeness and to 16.1% with 53% completeness by removing some outliers. In the high-dimensional space, machine learning algorithms are used with input patterns (u-g, g-r, r-i, i-z), (u-g, g-r, r-i, i-z, r), (NUV-u, u-g, g-r, r-i, i-z) and (NUV-u, u-g, g-r, r-i, i-z, r). RR Lyrae stars, which belong to the class of interest in our paper, are rare compared to other stars. For the highly imbalanced data, cost-sensitive Support Vector Machine (SVM), cost-sensitive Random Forest and Fast Boxes are used. The results show that information from GALEX is helpful for identifying RR Lyrae stars and Fast Boxes are the best performers on the skewed data in our case.

astro-ph.IM↗

arXiv2017

Improving Neural Machine Translation through Phrase-based Forced Decoding

Compared to traditional statistical machine translation (SMT), neural machine translation (NMT) often sacrifices adequacy for the sake of fluency. We propose a method to combine the advantages of traditional SMT and NMT by exploiting an existing phrase-based SMT model to compute the phrase-based decoding cost for an NMT output and then using this cost to rerank the n-best NMT outputs. The main challenge in implementing this approach is that NMT outputs may not be in the search space of the standard phrase-based decoding algorithm, because the search space of phrase-based SMT is limited by the phrase-based translation rule table. We propose a soft forced decoding algorithm, which can always successfully find a decoding path for any NMT output. We show that using the forced decoding cost to rerank the NMT outputs can successfully improve translation quality on four different language pairs.

cs.CL↗

arXiv2017

Symmetric Space-times from Thermodynamics

We apply thermodynamics method to generate exact solution with maximum symmetric surface for Einstein equation without solving it. The exact solutions are identified with which people have solved before. The horizons structure of solutions are discussed in different situation. These results suggest that plane,spherical,pseudo-spherical symmetric spacetime form a family of solutions, and this relationship can also be extended to Kerr spacetime and Taub-NUT spacetime. It sheds light on exploring more general definition of Misner-Sharp mass.

gr-qc↗

arXiv2012

Canonical Ensemble Model for the Black Hole Quantum Tunneling Radiation

In this paper, a canonical ensemble model for the black hole quantum tunneling radiation is introduced. With this model the probability distribution function corresponding to the emission shell is calculated. Comparing with this function, the statistical significance of the quantum tunneling radiation spectrum of black holes is investigated. Moreover, by calculating the entropy of the emission shell, a discussion about the mechanism of information flowing out from the black hole is given too.

hep-th↗

arXiv2010

Discussion on Event Horizon and Quantum Ergosphere of Evaporating Black Holes in a Tunnelling Framework

In this paper, with the Parikh-Wilczek tunnelling framework the positions of the event horizon of the Vaidya black hole and the Vaidya-Bonner black hole are calculated respectively. We find that the event horizon and the apparent horizon of these two black holes correspond respectively to the two turning points of the Hawking radiation tunnelling barrier. That is, the quantum ergosphere coincides with the tunnelling barrier. Our calculation also implies that the Hawking radiation comes from the apparent horizon.

hep-th↗

arXiv2008

Black hole quantum tunnelling and black hole entropy correction

Parikh-Wilczek tunnelling framework, which treats Hawking radiation as a tunnelling process, is investigated again. As the first order correction, the log-corrected entropy-area relation naturally emerges in the tunnelling picture if we consider the emission of a spherical shell. The second order correction of the emission rate for the Schwarzschild black hole is calculated too. In this level, the result is still in agreement with the unitary theory, however, the entropy of the black hole will contain three parts: the usual Bekenstein-Hawking entropy, the logarithmic term and the inverse area term. In our results the coefficient of the logarithmic term is -1. Apart from a coefficient, Our correction to the black hole entropy is consistent with that of loop quantum gravity.

hep-th↗

arXiv2006

Massive uncharged and charged particles' tunneling from the Horowitz-Strominger Dilaton black hole

Originally, Parikh and Wilczek's work is only suitable for the massless particles' tunneling. But their work has been further extended to the cases of massive uncharged and charged particles' tunneling recently. In this paper, as a particular black hole solution, we apply this extended method to reconsider the tunneling effect of the H.S Dilaton black hole. We investigate the behavior of both massive uncharged and charged particles, and respectively calculate the emission rate at the event horizon. Our result shows that their emission rates are also consistent with the unitary theory. Moreover, comparing with the case of massless particles' tunneling, we find that this conclusion is independent of the kind of particles. And it is probably caused by the underlying relationship between this method and the laws of black hole thermodynamics.

gr-qc↗

arXiv2006

Massive particles' Hawking radiation via tunneling from the G.H Dilaton black hole

In the past, Hawking radiation was viewed as a tunneling process and the barrier was just created by the outgoing particle itself. In this paper, Parikh's recent work is extended to the case of massive particles' tunneling. We investigate the behavior of the tunneling massive particles from a particular black hole solution-G.H Dilaton black hole which is obtained from the string theory, and calculate the emission rate at which massive particles tunnel across the event horizon. We obtain that the result is also consistent with an underlying unitary theory. Furthermore, the result takes the same functional form as that of massless particles.

gr-qc↗

arXiv2006

Tunnelling Effect and Hawking Radiation from a Vaidya Black Hole

In this paper, we extend Parikh' work to the non-stationary black hole. As an example of the non-stationary black hole, we study the tunnelling effect and Hawking radiation from a Vaidya black hole whose Bondi mass is identical to its mass parameter. We view Hawking radiation as a tunnelling process across the event horizon and calculate the tunnelling probability. We find that the result is different from Parikh's work because $\frac{dr_{H}}{dv}$ is the function of Bondi mass m(v).

gr-qc↗

arXiv2006

The relation between Hawking radiation via tunnelling and the laws of black hole thermodynamics

In Parikh and Wilczek's original works, the laws of black hole thermodynamics are not referred and it seems that there is no relation between Hawking radiation via tunnelling and the laws of black hole thermodynamics in their works. However, taking examples for the R-N black hole and the Kerr black hole, we find that they are correlated and even consistent if the tunnelling process is a reversible process.

gr-qc↗

arXiv2006

Information Loss in Black Hole Evaporation

Parikh-Wilczek tunnelling framework is investigated again. We argue that Parikh-Wilczek's treatment, which satisfies the first law of black hole thermodynamics and consists with an underlying unitary theory, is only suitable for a reversible process. Because of the negative heat capacity, an evaporating black hole is a highly unstable system. That is, the factual emission process is irreversible, the unitary theory will not be satisfied and the information loss is possible.

hep-th↗