arXiv Science⌕ Search

arXiv subjects

Wei Dai

Publications and source records attributed to Wei Dai.

At least 289 records · Page 16Linked to original sources
arXiv2016

On $\mathbb{Z}_{2}\mathbb{Z}_{2}[u]$-$(1+u)$-additive constacyclic

In this paper, we study $\mathbb{Z}_{2}\mathbb{Z}_{2}[u]$-$(1+u)$-additive constacyclic code of arbitrary length. Firstly, we study the algebraic structure of this family of codes and a set of generator polynomials for this family as a $(\mathbb{Z}_{2}+u\mathbb{Z}_{2})[x]$-submodule of the ring $R_{α,β}$. Secondly, we give the minimal generating sets of this family codes, and we determine the relationship of generators between the $\mathbb{Z}_{2}\mathbb{Z}_{2}[u]$-$(1+u)$-additive constacyclic codes and its dual and give the parameters in terms of the degrees of the generator polynomials of the code. Lastly, we also study $\mathbb{Z}_{2}\mathbb{Z}_{2}[u]$-$(1+u)$-additive constacyclic code in terms of the Gray images.

cs.IT↗
arXiv2016

Very Deep Convolutional Neural Networks for Raw Waveforms

Learning acoustic models directly from the raw waveform data with minimal processing is challenging. Current waveform-based models have generally used very few (~2) convolutional layers, which might be insufficient for building high-level discriminative features. In this work, we propose very deep convolutional neural networks (CNNs) that directly use time-domain waveforms as inputs. Our CNNs, with up to 34 weight layers, are efficient to optimize over very long sequences (e.g., vector of size 32000), necessary for processing acoustic waveforms. This is achieved through batch normalization, residual learning, and a careful design of down-sampling in the initial layers. Our networks are fully convolutional, without the use of fully connected layers and dropout, to maximize representation learning. We use a large receptive field in the first convolutional layer to mimic bandpass filters, but very small receptive fields subsequently to control the model capacity. We demonstrate the performance gains with the deeper models. Our evaluation shows that the CNN with 18 weight layers outperform the CNN with 3 weight layers by over 15% in absolute accuracy for an environmental sound recognition task and matches the performance of models using log-mel features.

cs.SD↗
arXiv2016

Structured Compressive Sensing Based Spatio-Temporal Joint Channel Estimation for FDD Massive MIMO

Massive MIMO is a promising technique for future 5G communications due to its high spectrum and energy efficiency. To realize its potential performance gain, accurate channel estimation is essential. However, due to massive number of antennas at the base station (BS), the pilot overhead required by conventional channel estimation schemes will be unaffordable, especially for frequency division duplex (FDD) massive MIMO. To overcome this problem, we propose a structured compressive sensing (SCS)-based spatio-temporal joint channel estimation scheme to reduce the required pilot overhead, whereby the spatio-temporal common sparsity of delay-domain MIMO channels is leveraged. Particularly, we first propose the non-orthogonal pilots at the BS under the framework of CS theory to reduce the required pilot overhead. Then, an adaptive structured subspace pursuit (ASSP) algorithm at the user is proposed to jointly estimate channels associated with multiple OFDM symbols from the limited number of pilots, whereby the spatio-temporal common sparsity of MIMO channels is exploited to improve the channel estimation accuracy. Moreover, by exploiting the temporal channel correlation, we propose a space-time adaptive pilot scheme to further reduce the pilot overhead. Additionally, we discuss the proposed channel estimation scheme in multi-cell scenario. Simulation results demonstrate that the proposed scheme can accurately estimate channels with the reduced pilot overhead, and it is capable of approaching the optimal oracle least squares estimator.

cs.IT↗
arXiv2016

Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework. Whenever possible, we perform computations asynchronously, which helps attain speedups on multicore machines as well as in distributed environments. Moreover, instead of worst-case bounded delays, our methods only depend (mildly) on \emph{expected} delays, allowing them to be robust to stragglers and faulty worker threads. Our algorithms assume block-separable constraints, and subsume the recent Block-Coordinate Frank-Wolfe (BCFW) method~\citep{lacoste2013block}. Our analysis reveals problem-dependent quantities that govern the speedups of our methods over BCFW. We present experiments on structural SVM and Group Fused Lasso, obtaining significant speedups over competing state-of-the-art (and synchronous) methods.

stat.ML↗
arXiv2016

Heavy Quark and Quarkonium Transport in High Energy Nuclear Collisions

The strong interaction between heavy quarks and the quark gluon plasma makes the open and hidden charm hadrons be sensitive probes of the deconfinement phase transition in high energy nuclear collisions. Both the cold and hot nuclear matter effects change with the colliding energy and significantly influence the heavy quark and charmonium yield and their transverse momentum distributions. The ratio of averaged quarkonium transverse momentum square and the elliptic flow reveal the nature of the QCD medium created in heavy ion collisions at SPS, RHIC and LHC energies.

hep-ph↗
arXiv2015

Strategies and Principles of Distributed Machine Learning on Big Data

The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics thereupon. In order to run ML algorithms at such scales, on a distributed cluster with 10s to 1000s of machines, it is often the case that significant engineering efforts are required --- and one might fairly ask if such engineering truly falls within the domain of ML research or not. Taking the view that Big ML systems can benefit greatly from ML-rooted statistical and algorithmic insights --- and that ML researchers should therefore not shy away from such systems design --- we discuss a series of principles and strategies distilled from our recent efforts on industrial-scale ML solutions. These principles and strategies span a continuum from application, to engineering, and to theoretical research and development of Big ML systems and architectures, with the goal of understanding how to make them efficient, generally-applicable, and supported with convergence and scaling guarantees. They concern four key questions which traditionally receive little attention in ML research: How to distribute an ML program over a cluster? How to bridge ML computation with inter-machine communication? How to perform such communication? What should be communicated between machines? By exposing underlying statistical and algorithmic characteristics unique to ML programs but not typically seen in traditional computer programs, and by dissecting successful cases to reveal how we have harnessed these principles to design and develop both high-performance distributed ML software as well as general-purpose ML frameworks, we present opportunities for ML researchers and practitioners to further shape and grow the area that lies between ML and systems.

stat.ML↗
arXiv2015

$η$ meson production of high-energy nuclear collisions at NLO

The transverse momentum spectrum of $η$ meson in relativistic heavy-ion collisions is studied at the next-to-leading-order (NLO) within the perturbative QCD, where the jet quenching effect in the QGP is incorporated with the effectively medium-modified $η$ fragmentation functions using the higher-twist approach. We show that the theoretical simulations could give nice descriptions of PHENIX data on $η$ meson in both $\rm p+p$ and central $\rm Au+Au$ collisions at the RHIC, and also provide numerical predictions of $η$ spectra in central $\rm Pb+Pb$ collisions with $\sqrt{s_{NN}}=2.76$~TeV at the LHC. The ratios of $η/π^0$ in $\rm p+p$ and in central $\rm Au+Au$ collisions at $200$~GeV are found to overlap in a wide $p_T$ region, which matches well the measured ratio $η/ π^0$ by PHENIX. We demonstrate that, at the asymptotic region when $p_{T} \rightarrow \infty$ the ratios of $η/π^{0}$ in both $\rm Au+Au$ and $\rm p+p$ are almost determined only by quark jets fragmentation and thus approach to the one in $e^{+} e^{-}$ scattering; in addition, the almost identical gluon (quark) contribution fractions to $η$ and to $π$ result in a rather moderate variation of $η/π^{0}$ distribution at intermediate and high $p_T$ region in $\rm A+A$ relative to that in $\rm p+p$; while a slightly higher $η/π^{0}$ at small $p_T$ in $\rm Au+Au$ can be observed due to larger suppression of gluon contribution fraction to $π^{0}$ as compared to the one to $η$. The theoretical prediction for $η/ π^0$ at the LHC has also been presented.

nucl-th↗
arXiv2015

Merge Frame Design for Video Stream Switching using Piecewise Constant Functions

The ability to efficiently switch from one pre-encoded video stream to another (e.g., for bitrate adaptation or view switching) is important for many interactive streaming applications. Recently, stream-switching mechanisms based on distributed source coding (DSC) have been proposed. In order to reduce the overall transmission rate, these approaches provide a "merge" mechanism, where information is sent to the decoder such that the exact same frame can be reconstructed given that any one of a known set of side information (SI) frames is available at the decoder (e.g., each SI frame may correspond to a different stream from which we are switching). However, the use of bit-plane coding and channel coding in many DSC approaches leads to complex coding and decoding. In this paper, we propose an alternative approach for merging multiple SI frames, using a piecewise constant (PWC) function as the merge operator. In our approach, for each block to be reconstructed, a series of parameters of these PWC merge functions are transmitted in order to guarantee identical reconstruction given the known side information blocks. We consider two different scenarios. In the first case, a target frame is first given, and then merge parameters are chosen so that this frame can be reconstructed exactly at the decoder. In contrast, in the second scenario, the reconstructed frame and merge parameters are jointly optimized to meet a rate-distortion criteria. Experiments show that for both scenarios, our proposed merge techniques can outperform both a recent approach based on DSC and the SP-frame approach in H.264, in terms of compression efficiency and decoder complexity.

cs.MM↗
arXiv2015

Structured Matching Pursuit for Reconstruction of Dynamic Sparse Channels

In this paper, by exploiting the special features of temporal correlations of dynamic sparse channels that path delays change slowly over time but path gains evolve faster, we propose the structured matching pursuit (SMP) algorithm to realize the reconstruction of dynamic sparse channels. Specifically, the SMP algorithm divides the path delays of dynamic sparse channels into two different parts to be considered separately, i.e., the common channel taps and the dynamic channel taps. Based on this separation, the proposed SMP algorithm simultaneously detects the common channel taps of dynamic sparse channels in all time slots at first, and then tracks the dynamic channel taps in each single time slot individually. Theoretical analysis of the proposed SMP algorithm provides a guarantee that the common channel taps can be successfully detected with a high probability, and the reconstruction distortion of dynamic sparse channels is linearly upper bounded by the noise power. Simulation results demonstrate that the proposed SMP algorithm has excellent reconstruction performance with competitive computational complexity compared with conventional reconstruction algorithms.

cs.IT↗
arXiv2015

Tracking A Dynamic Sparse Channel Via Differential Orthogonal Matching Pursuit

This paper considers the problem of tracking a dynamic sparse channel in a broadband wireless communication system. A probabilistic signal model is firstly proposed to describe the special features of temporal correlations of dynamic sparse channels: path delays change slowly over time, while path gains evolve faster. Based on such temporal correlations, we then propose the differential orthogonal matching pursuit (D-OMP) algorithm to track a dynamic sparse channel in a sequential way by updating the small channel variation over time. Compared with other channel tracking algorithms, simulation results demonstrate that the proposed D-OMP algorithm can track dynamic sparse channels faster with improved accuracy.

cs.IT↗
arXiv2015

Petuum: A New Platform for Distributed Machine Learning on Big Data

What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)? Modern parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-synchronous processing paradigm popularized by MapReduce, or even specialized graph-based execution that relies on graph representations of ML programs. The variety of approaches tends to pull systems and algorithms design in different directions, and it remains difficult to find a universal platform applicable to a wide range of ML programs at scale. We propose a general-purpose framework that systematically addresses data- and model-parallel challenges in large-scale ML, by observing that many ML programs are fundamentally optimization-centric and admit error-tolerant, iterative-convergent algorithmic solutions. This presents unique opportunities for an integrative system design, such as bounded-error network synchronization and dynamic scheduling based on ML program structure. We demonstrate the efficacy of these system designs versus well-known implementations of modern ML algorithms, allowing ML programs to run in much less time and at considerably larger model sizes, even on modestly-sized compute clusters.

stat.ML↗
arXiv2014

LightLDA: Big Topic Models on Modest Compute Clusters

When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized clusters with thousands of nodes, which are out of reach for most practitioners or academic researchers. We consider this challenge in the context of topic modeling on web-scale corpora, and show that with a modest cluster of as few as 8 machines, we can train a topic model with 1 million topics and a 1-million-word vocabulary (for a total of 1 trillion parameters), on a document collection with 200 billion tokens -- a scale not yet reported even with thousands of machines. Our major contributions include: 1) a new, highly efficient O(1) Metropolis-Hastings sampling algorithm, whose running cost is (surprisingly) agnostic of model size, and empirically converges nearly an order of magnitude faster than current state-of-the-art Gibbs samplers; 2) a structure-aware model-parallel scheme, which leverages dependencies within the topic model, yielding a sampling strategy that is frugal on machine memory and network communication; 3) a differential data-structure for model storage, which uses separate data structures for high- and low-frequency words to allow extremely large models to fit in memory, while maintaining high inference speed; and 4) a bounded asynchronous data-parallel scheme, which allows efficient distributed processing of massive data via a parameter server. Our distribution strategy is an instance of the model-and-data-parallel programming model underlying the Petuum framework for general distributed ML, and was implemented on top of the Petuum open-source system. We provide experimental evidence showing how this development puts massive models within reach on a small cluster while still enjoying proportional time cost reductions with increasing cluster size, in comparison with alternative options.

stat.ML↗
arXiv2014

High-Performance Distributed ML at Scale through Parameter Server Consistency Models

As Machine Learning (ML) applications increase in data size and model complexity, practitioners turn to distributed clusters to satisfy the increased computational and memory demands. Unfortunately, effective use of clusters for ML requires considerable expertise in writing distributed code, while highly-abstracted frameworks like Hadoop have not, in practice, approached the performance seen in specialized ML implementations. The recent Parameter Server (PS) paradigm is a middle ground between these extremes, allowing easy conversion of single-machine parallel ML applications into distributed ones, while maintaining high throughput through relaxed "consistency models" that allow inconsistent parameter reads. However, due to insufficient theoretical study, it is not clear which of these consistency models can really ensure correct ML algorithm output; at the same time, there remain many theoretically-motivated but undiscovered opportunities to maximize computational throughput. Motivated by this challenge, we study both the theoretical guarantees and empirical behavior of iterative-convergent ML algorithms in existing PS consistency models. We then use the gleaned insights to improve a consistency model using an "eager" PS communication mechanism, and implement it as a new PS system that enables ML algorithms to reach their solution more quickly.

cs.LG↗
arXiv2014

$L^{p}$ estimates for the bilinear Hilbert transform for $1/2<p\leq2/3$: A counterexample and generalizations to non-smooth symbols

M. Lacey and C. Thiele proved in [27] (Annals of Math. (1997)) and [28] (Annals of Math. (1999)) that the bilinear Hilbert transform maps $L^{p_1}\times L^{p_2}\rightarrow L^{p}$ boundedly when $\frac{1}{p_1}+\frac{1}{p_2}=\frac{1}{p}$ with $1<p_{1}, \, p_{2}\leq\infty$ and $\frac{2}{3}<p<\infty$. Whether the $L^p$ estimates hold in the range $p\in (1/2,2/3]$ has remained an open problem since then. In this paper, we prove that the bilinear Hilbert transform does not map $\mathcal{F}L^{p'_{1}}\times L^{p_{2}}\rightarrow L^{p}$ for $p_1<2$ and $L^{p_{1}}\times \mathcal{F}L^{p'_{2}}\rightarrow L^{p}$ for $p_2<2$ boundedly (Theorem 1.2). In particular, this shows that the bilinear Hilbert transform neither maps $\mathcal{F}L^{p'_{1}}\times L^{p_{2}}\rightarrow L^{p}$ nor $L^{p_{1}}\times \mathcal{F}L^{p'_{2}}\rightarrow L^{p}$ for $\frac{1}{2}<p<\frac{2}{3}$. Nevertheless, we can establish $L^p$ estimates for the bilinear Fourier multipliers whose symbols are not identical to but arbitrarily close to that of the bilinear Hilbert transform in the full range $p\in(1/2,\infty)$ (Theorem 1.3).

math.CA↗
arXiv2014

Power Allocation in Compressed Sensing of Non-uniformly Sparse Signals

This paper studies the problem of power allocation in compressed sensing when different components in the unknown sparse signal have different probability to be non-zero. Given the prior information of the non-uniform sparsity and the total power budget, we are interested in how to optimally allocate the power across the columns of a Gaussian random measurement matrix so that the mean squared reconstruction error is minimized. Based on the state evolution technique originated from the work by Donoho, Maleki, and Montanari, we revise the so called approximate message passing (AMP) algorithm for the reconstruction and quantify the MSE performance in the asymptotic regime. Then the closed form of the optimal power allocation is obtained. The results show that in the presence of measurement noise, uniform power allocation, which results in the commonly used Gaussian random matrix with i.i.d. entries, is not optimal for non-uniformly sparse signals. Empirical results are presented to demonstrate the performance gain.

cs.IT↗
arXiv2014

$L^{p}$ estimates for bilinear and multi-parameter Hilbert transforms

C. Muscalu, J. Pipher, T. Tao and C. Thiele proved in \cite{MPTT1} that the standard bilinear and bi-parameter Hilbert transform does not satisfy any $L^{p}$ estimates. They also raised a question asking if a bilinear and bi-parameter multiplier operator defined by $$ T_{m}(f_{1},f_{2})(x):=\int_{\mathbb{R}^{4}}m(ξ,η)\hat{f_{1}}(ξ_{1},η_{1})\hat{f_{2}}(ξ_{2},η_{2})e^{2πix\cdot((ξ_{1},η_{1})+(ξ_{2},η_{2}))}dξdη$$ satisfies any $L^p$ estimates, where the symbol $m$ satisfies $$ |\partial_ξ^α\partial_η^βm(ξ,η)|\lesssim\frac{1}{dist(ξ,Γ_{1})^{|α|}}\cdot\frac{1}{dist(η,Γ_{2})^{|β|}} $$ for sufficiently many multi-indices $α=(α_{1},α_{2})$ and $β=(β_{1},β_{2})$, $Γ_{i}$ ($i=1,2$) are subspaces in $\mathbb{R}^{2}$ and $dim \, Γ_{1}=0, \, dim \, Γ_{2}=1$. P. Silva answered partially this question in \cite{S} and proved that $T_{m}$ maps $L^{p_1}\times L^{p_2}\rightarrow L^{p}$ boundedly when $\frac{1}{p_1}+\frac{1}{p_2}=\frac{1}{p}$ with $p_1, p_2>1$, $\frac{1}{p_1}+\frac{2}{p_2}<2$ and $\frac{1}{p_2}+\frac{2}{p_1}<2$. One observes that the admissible range here for these tuples $(p_1,p_2,p)$ is a proper subset contained in the admissible range of BHT. In this paper, we establish the same $L^{p}$ estimates as BHT in the full range for the bilinear and multi-parameter Hilbert transforms with arbitrary symbols satisfying appropriate decay assumptions (Theorem 1.3). Moreover, we also establish the same $L^p$ estimates as BHT for certain modified bilinear and bi-parameter Hilbert transforms with $dim \, Γ_{1}=dim \, Γ_{2}=1$ but with a slightly better decay than that for the bilinear and bi-parameter Hilbert transform (Theorem 1.4).

math.CA↗
arXiv2013

Consistent Bounded-Asynchronous Parameter Servers for Distributed ML

In distributed ML applications, shared parameters are usually replicated among computing nodes to minimize network overhead. Therefore, proper consistency model must be carefully chosen to ensure algorithm's correctness and provide high throughput. Existing consistency models used in general-purpose databases and modern distributed ML systems are either too loose to guarantee correctness of the ML algorithms or too strict and thus fail to fully exploit the computing power of the underlying distributed system. Many ML algorithms fall into the category of \emph{iterative convergent algorithms} which start from a randomly chosen initial point and converge to optima by repeating iteratively a set of procedures. We've found that many such algorithms are to a bounded amount of inconsistency and still converge correctly. This property allows distributed ML to relax strict consistency models to improve system performance while theoretically guarantees algorithmic correctness. In this paper, we present several relaxed consistency models for asynchronous parallel computation and theoretically prove their algorithmic correctness. The proposed consistency models are implemented in a distributed parameter server and evaluated in the context of a popular ML application: topic modeling.

stat.ML↗