arXiv subjects

Fei Wang

Publications and source records attributed to Fei Wang.

At least 739 records · Page 41Linked to original sources

arXiv2017

Accelerate RNN-based Training with Importance Sampling

Importance sampling (IS) as an elegant and efficient variance reduction (VR) technique for the acceleration of stochastic optimization problems has attracted many researches recently. Unlike commonly adopted stochastic uniform sampling in stochastic optimizations, IS-integrated algorithms sample training data at each iteration with respect to a weighted sampling probability distribution $P$, which is constructed according to the precomputed importance factors. Previous experimental results show that IS has achieved remarkable progresses in the acceleration of training convergence. Unfortunately, the calculation of the sampling probability distribution $P$ causes a major limitation of IS: it requires the input data to be well-structured, i.e., the feature vector is properly defined. Consequently, recurrent neural networks (RNN) as a popular learning algorithm is not able to enjoy the benefits of IS due to the fact that its raw input data, i.e., the training sequences, are often unstructured which makes calculation of $P$ impossible. In considering of the the popularity of RNN-based learning applications and their relative long training time, we are interested in accelerating them through IS. This paper propose a novel Fast-Importance-Mining algorithm to calculate the importance factor for unstructured data which makes the application of IS in RNN-based applications possible. Our experimental evaluation on popular open-source RNN-based learning applications validate the effectiveness of IS in improving the convergence rate of RNNs.

cs.LG↗

arXiv2017

Probing Lepton Flavor Violation Signal via γγ\to l_i\barl_j in the Left-Right Twin Higgs Model at the ILC

To explain the small neutrino masses, heavy Majorana neutrinos are introduced in the left-right twin Higgs model. The heavy neutrinos, together with the charged scalars and the heavy gauge bosons, may contribute large mixings between the neutrinos and the charged leptons, which may induce some distinct lepton flavor violating processes. We will check the \bar \ell_i \ell_j (i,j= e,μ,τ,i\neq j) productions in the γγcollision in the left-right twin Higgs model, and find that the production rates may be large in some specific parameter space, in the optimal cases even possible to be detected with reasonable kinematical cuts. we have also shown that these collisions can constrain effectively the model parameters such as the Higgs vacuum expectation value and the right-handed neutrino mass, etc., and may serve as a sensitive probe of this new physics model.

hep-ph↗

arXiv2017

Probing degenerate heavy Higgs bosons in NMSSM with vector-like particles

In this work, we investigate the degenerate heavy Higgs bosons in the Next-to-Minimal Supersymmetric Standard Model (NMSSM) by introducing vector-like particles. Such an extension is well motivated from the top-down view since some grand unified theories usually predict the existence of singlet scalars and vector-like particles at weak scale. Under the constraints from the LHC and dark matter experiments, we find that (1) the null results of searching for high mass resonances have tightly constrained the parameter space; (2) two degenerate heavy singlet Higgs bosons $h_2$ and $a_1$ can sizably decay to $χ^0_1χ^0_1$ invisibly. Therefore, search for the monojet events through the process $gg \to h_2 / a_1 (\to χ^0_1 χ^0_1) j$ may further test our scenario at the future LHC.

hep-ph↗

arXiv2017

Solving The Muon g-2 Anomaly With Natural NMSSM From Generalized Deflected AMSB

We propose to realize (natural) NMSSM spectrum from deflected AMSB with new messenger-matter interactions. With additional messenger-matter interactions involving ${\bf 10}\oplus{\bf \overline{10}}$ representation messengers, the muon g-2 anomaly can be solved at $2σ$ (or $3σ$) level with the corresponding gluino mass range $2.8~{\rm TeV}<m_{\tilde{g}}<5.4~ {\rm TeV}$ (or $2.6 ~{\rm TeV}<m_{\tilde{g}}<7.3~ {\rm TeV}$). Besides, our scenario is fairly natural within which the involved fine tuning can be as low as 47. So, in the framework of AMSB-type scenarios, NMSSM can be advantageous in explaining the muon g-2 anomaly in compare with MSSM.

hep-ph↗

arXiv2017

GaDei: On Scale-up Training As A Service For Deep Learning

Deep learning (DL) training-as-a-service (TaaS) is an important emerging industrial workload. The unique challenge of TaaS is that it must satisfy a wide range of customers who have no experience and resources to tune DL hyper-parameters, and meticulous tuning for each user's dataset is prohibitively expensive. Therefore, TaaS hyper-parameters must be fixed with values that are applicable to all users. IBM Watson Natural Language Classifier (NLC) service, the most popular IBM cognitive service used by thousands of enterprise-level clients around the globe, is a typical TaaS service. By evaluating the NLC workloads, we show that only the conservative hyper-parameter setup (e.g., small mini-batch size and small learning rate) can guarantee acceptable model accuracy for a wide range of customers. We further justify theoretically why such a setup guarantees better model convergence in general. Unfortunately, the small mini-batch size causes a high volume of communication traffic in a parameter-server based system. We characterize the high communication bandwidth requirement of TaaS using representative industrial deep learning workloads and demonstrate that none of the state-of-the-art scale-up or scale-out solutions can satisfy such a requirement. We then present GaDei, an optimized shared-memory based scale-up parameter server design. We prove that the designed protocol is deadlock-free and it processes each gradient exactly once. Our implementation is evaluated on both commercial benchmarks and public benchmarks to demonstrate that it significantly outperforms the state-of-the-art parameter-server based implementation while maintaining the required accuracy and our implementation reaches near the best possible runtime performance, constrained only by the hardware limitation. Furthermore, to the best of our knowledge, GaDei is the only scale-up DL system that provides fault-tolerance.

stat.ML↗

arXiv2017

Solving the muon g-2 anomaly in deflected AMSB with messenger-matter interactions

We proposed to introduce general messenger-matter interactions in the deflected anomaly mediated SUSY breaking scenario to explain the $g_μ-2$ anomaly. Scenarios with complete or incomplete GUT multiplet messengers are discussed, respectively. The introduction of incomplete GUT mulitiplets can be advantageous in various aspects. We found that the $g_μ-2$ anomaly can be solved in both scenarios under current constraints including the gluino mass bounds, while the scenarios with incomplete GUT representation messengers are more favored by the $g_μ-2$ data. We also found that the gluino is upper bounded by about 2.5 TeV (2.0 TeV) in Scenario A and 3.0 TeV (2.7 TeV) in Scenario B if the generalized deflected AMSB scenarios are used to fully account for the $g_μ-2$ anomaly at $3σ$ ($2σ$) level. Such a gluino should be accessible in the future LHC searches. Dark matter constraints, including DM relic density and direct detection bounds, favor the scenario B with incomplete GUT multiplets. Much of the allowed parameter space for the scenario B could be covered by the future DM direct detection experiments.

hep-ph↗

arXiv2017

Global regularity for a family of 3D models of the axisymmetric Navier-Stokes equations

We consider a family of 3D models for the axi-symmetric incompressible Navier-Stokes equations. The models are derived by changing the strength of the convection terms in the axisymmetric Navier-Stokes equations written using a set of transformed variables. We prove the global regularity of the family of models in the case that the strength of convection is slightly stronger than that of the original Navier-Stokes equations, which demonstrates the potential stabilizing effect of convection.

math.AP↗

arXiv2017

Maximum Correntropy Adaptive Filtering Approach for Robust Compressive Sensing Reconstruction

Robust compressive sensing(CS) reconstruction has become an attractive research topic in recent years. Robust CS aims to reconstruct the sparse signals under non-Gaussian(i.e. heavy tailed) noises where traditional CS reconstruction algorithms may perform very poorly due to utilizing $l_2$ norm of the residual vector in optimization. Most of existing robust CS reconstruction algorithms are based on greedy pursuit method or convex relaxation approach. Recently, the adaptive filtering framework has been introduced to deal with the CS reconstruction, which shows desirable performance in both efficiency and reconstruction performance under Gaussian noise. In this paper, we propose an adaptive filtering based robust CS reconstruction algorithm, called $l_0$ regularized maximum correntropy criterion($l_0$-MCC) algorithm, which combines the adaptive filtering framework and maximum correntropy criterion(MCC). MCC has recently been successfully used in adaptive filtering due to its robustness to impulsive non-Gaussian noises and low computational complexity. We analyze theoretically the stability of the proposed $l_0$-MCC algorithm. A mini-batch based $l_0$-MCC(MB-$l_0$-MCC) algorithm is further developed to speed up the convergence. Comparison with existing robust CS reconstruction algorithms is conducted via simulations, showing that the proposed $l_0$-MCC and MB-$l_0$-MCC can achieve significantly better performance than other algorithms.

cs.IT↗

arXiv2017

Residual Attention Network for Image Classification

In this work, we propose "Residual Attention Network", a convolutional neural network using attention mechanism which can incorporate with state-of-art feed forward network architecture in an end-to-end training fashion. Our Residual Attention Network is built by stacking Attention Modules which generate attention-aware features. The attention-aware features from different modules change adaptively as layers going deeper. Inside each Attention Module, bottom-up top-down feedforward structure is used to unfold the feedforward and feedback attention process into a single feedforward process. Importantly, we propose attention residual learning to train very deep Residual Attention Networks which can be easily scaled up to hundreds of layers. Extensive analyses are conducted on CIFAR-10 and CIFAR-100 datasets to verify the effectiveness of every module mentioned above. Our Residual Attention Network achieves state-of-the-art object recognition performance on three benchmark datasets including CIFAR-10 (3.90% error), CIFAR-100 (20.45% error) and ImageNet (4.8% single model and single crop, top-5 error). Note that, our method achieves 0.6% top-1 accuracy improvement with 46% trunk depth and 69% forward FLOPs comparing to ResNet-200. The experiment also demonstrates that our network is robust against noisy labels.

cs.CV↗

arXiv2017

Google Map Aided Visual Navigation for UAVs in GPS-denied Environment

We propose a framework for Google Map aided UAV navigation in GPS-denied environment. Geo-referenced navigation provides drift-free localization and does not require loop closures. The UAV position is initialized via correlation, which is simple and efficient. We then use optical flow to predict its position in subsequent frames. During pose tracking, we obtain inter-frame translation either by motion field or homography decomposition, and we use HOG features for registration on Google Map. We employ particle filter to conduct a coarse to fine search to localize the UAV. Offline test using aerial images collected by our quadrotor platform shows promising results as our approach eliminates the drift in dead-reckoning, and the small localization error indicates the superiority of our approach as a supplement to GPS.

cs.CV↗

arXiv2017

SPARTan: Scalable PARAFAC2 for Large & Sparse Data

In exploratory tensor mining, a common problem is how to analyze a set of variables across a set of subjects whose observations do not align naturally. For example, when modeling medical features across a set of patients, the number and duration of treatments may vary widely in time, meaning there is no meaningful way to align their clinical records across time points for analysis purposes. To handle such data, the state-of-the-art tensor model is the so-called PARAFAC2, which yields interpretable and robust output and can naturally handle sparse data. However, its main limitation up to now has been the lack of efficient algorithms that can handle large-scale datasets. In this work, we fill this gap by developing a scalable method to compute the PARAFAC2 decomposition of large and sparse datasets, called SPARTan. Our method exploits special structure within PARAFAC2, leading to a novel algorithmic reformulation that is both fast (in absolute time) and more memory-efficient than prior work. We evaluate SPARTan on both synthetic and real datasets, showing 22X performance gains over the best previous implementation and also handling larger problem instances for which the baseline fails. Furthermore, we are able to apply SPARTan to the mining of temporally-evolving phenotypes on data taken from real and medically complex pediatric patients. The clinical meaningfulness of the phenotypes identified in this process, as well as their temporal evolution over time for several patients, have been endorsed by clinical experts.

cs.LG↗

arXiv2017

Ionic vibration induced transparency and Autler-Townes splitting

In this work, the absorption spectrum of a two-level ion in a linear Paul trap is investigated, the ion is supposed to be driven by two orthogonal laser beams, the one along the axial of the trap acts as the control light beam, the other as probe beam. When the frequency of the control laser is tuned to the first red sideband of the ionic transition, the coupling between the internal states of the ion and vibrational mode turns out to be a Jaynes-Cummings (JC) Hamiltonian, which together with the coupling between the probe beam and the two-level ion constructs a Λ-type three-level structure. In this case the transparency window may appear in the absorption spectrum of the probe light, which is induced by the ionic vibration and is very similar to the cavity induced transparency [1996 Opt. Commun. 126 230-235]. On the other hand, when the frequency of the control laser is tuned to the first blue sideband of the ionic transition, the two-level ion and vibrational mode are governed by an anti-Jaynes-Cummings (anti-JC) Hamiltonian, the total system including the probe beam forms a V -type threelevel structure. And the Autler-Townes splitting in the absorption spectrum is found.

quant-ph↗

arXiv2017

Thermal convection with mixed thermal boundary conditions: Effects of insulating lids at the top

The effects of insulating lids on the convection beneath were investigated experimentally using rectangular convection cells in the flux Rayleigh number range $2.3\times10^{9}\leq Ra_F \leq 1.8\times10^{11}$ and cylindrical cells in the range $1.4\times10^{10}\leq Ra_F \leq 1.2\times10^{12}$ with the Prandtl number Pr fixed at 4.3. It is found that the presence of the insulating lids leads to reduction of the global heat transfer efficiency as expected, which primarily depends on the insulating area but is insensitive to the detailed insulating patterns. At the leading order level, the magnitude of temperature fluctuation in the bulk fluid is, again, found to be insensitive to the insulating pattern and mainly depends on the insulating area; while the temperature probability density function (PDF) in the bulk is essentially invariant with respect to both insulating area and the spatial pattern of the lids. The flow dynamics, on the other hand, is sensitive to both the covering area and the spatial distribution of the lids. At fixed $Ra_F$, the flow strength is found to increase with increasing insulating area so as to transfer the same amount of heat through a smaller cooling area. Moreover, for a constant insulating area, a symmetric insulating pattern results in a symmetric flow pattern, i.e. double-roll structure; whereas asymmetric insulating pattern leads to asymmetric flow, i.e. single-roll structure. It is further found that the symmetry breaking of the insulating pattern leads to a stronger flow that enhances the horizontal velocity more than the vertical one.

physics.flu-dyn↗

arXiv2016

Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study

This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and number of learners on runtime performance and model accuracy. We introduce a new learning rate modulation strategy to counter the effect of stale gradients and propose a new synchronization protocol that can effectively bound the staleness in gradients, improve runtime performance and achieve good model accuracy. Our empirical investigation reveals a principled approach for distributed training of neural networks: the mini-batch size per learner should be reduced as more learners are added to the system to preserve the model accuracy. We validate this approach using commonly-used image classification benchmarks: CIFAR10 and ImageNet.

stat.ML↗

arXiv2016

Computing the generators of the truncated real radical ideal by moment matrices and SDP facial reduction

Recent breakthroughs have been made in the use of semidefinite programming and its application to real polynomial solving. For example, the real radical of a zero dimensional ideal, can be determined by such approaches as shown by Lasserre and collaborators. Some progress has been made on the determination of the real radical in positive dimension by Ma, Wang and Zhi. Such work involves the determination of maximal rank semidefinite moment matrices. Existing methods are computationally expensive and have poorer accuracy on larger examples. This paper is motivated by problems in the numerical computation of the real radical ideal in the general positive case. In this paper we give a method to compute the generators of the real radical for any given degree $d$. We combine the use of moment matrices and techniques from SDP optimization: facial reduction first developed by Borwein and Wolkowicz. In use of the semidefinite moment matrices to compute the real radical, the maximum rank property is very key, and with facial reduction, it can be guaranteed with very high accuracy. Our algorithm can be used to test the real radical membership of a given polynomial. In a special situation, we can determine the real radical ideal in the positive dimensional case.

math.OC↗

arXiv2016

Economic information from Smart Meter: Nexus Between Demand Profile and Electricity Retail Price Between Demand Profile and Electricity Retail Price

In this paper, we demonstrate that a consumer's marginal system impact is only determined by their demand profile rather than their demand level. Demand profile clustering is identical to cluster consumers according to their marginal impacts on system costs. A profile-based uniform-rate price is economically efficient as real-time pricing. We develop a criteria system to evaluate the economic efficiency of an implemented retail price scheme in a distribution system by comparing profile clustering and daily-average clustering. Our criteria system can examine the extent of a retail price scheme's inefficiency even without information about the distribution system's daily cost structure. We analyze data from a real distribution system in China. In this system, targeting each consumer's high-impact days is more efficient than target high-impact consumers.

econ.GN↗

arXiv2016

The Sobolev stability threshold for 2D shear flows near Couette

We consider the 2D Navier-Stokes equation on $\mathbb T \times \mathbb R$, with initial datum that is $\varepsilon$-close in $H^N$ to a shear flow $(U(y),0)$, where $\| U(y) - y\|_{H^{N+4}} \ll 1$ and $N>1$. We prove that if $\varepsilon \ll ν^{1/2}$, where $ν$ denotes the inverse Reynolds number, then the solution of the Navier-Stokes equation remains $\varepsilon$-close in $H^1$ to $(e^{t ν\partial_{yy}}U(y),0)$ for all $t>0$. Moreover, the solution converges to a decaying shear flow for times $t \gg ν^{-1/3}$ by a mixing-enhanced dissipation effect, and experiences a transient growth of gradients. In particular, this shows that the stability threshold in finite regularity scales no worse than $ν^{1/2}$ for 2D shear flows close to the Couette flow.

math.AP↗

arXiv2016

On the local existence for an active scalar equation in critical regularity setting

In this note, we address the local well-posedness for the active scalar equation $\partial_t θ+ u\cdot \nabla θ=0$, where $u = - \nabla^\perp(-Δ)^{-1+β/2}θ$. The local existence of solutions in the Sobolev class $H^{1+β+ε}$, where $ε>0$ and $β\in (1,2)$, has been recently addressed in \cite{HKZ}. The critical case $ε=0$ has remained open. Using a different technique, we prove the local well-posedness in the Besov space $B^{1+β}_{2,1}$, where $β\in (1,2)$. The proof is based on log-Lipschitz estimates for the transport equation.

math.AP↗