arXiv subjects

Jian Li

Publications and source records attributed to Jian Li.

At least 613 records · Page 34Linked to original sources

arXiv2018

A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization

We analyze stochastic gradient algorithms for optimizing nonconvex, nonsmooth finite-sum problems. In particular, the objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a possibly non-differentiable but convex component. We propose a proximal stochastic gradient algorithm based on variance reduction, called ProxSVRG+. Our main contribution lies in the analysis of ProxSVRG+. It recovers several existing convergence results and improves/generalizes them (in terms of the number of stochastic gradient oracle calls and proximal oracle calls). In particular, ProxSVRG+ generalizes the best results given by the SCSG algorithm, recently proposed by [Lei et al., 2017] for the smooth nonconvex case. ProxSVRG+ is also more straightforward than SCSG and yields simpler analysis. Moreover, ProxSVRG+ outperforms the deterministic proximal gradient descent (ProxGD) for a wide range of minibatch sizes, which partially solves an open problem proposed in [Reddi et al., 2016b]. Also, ProxSVRG+ uses much less proximal oracle calls than ProxSVRG [Reddi et al., 2016b]. Moreover, for nonconvex functions satisfied Polyak-Łojasiewicz condition, we prove that ProxSVRG+ achieves a global linear convergence rate without restart unlike ProxSVRG. Thus, it can \emph{automatically} switch to the faster linear convergence in some regions as long as the objective function satisfies the PL condition locally in these regions. ProxSVRG+ also improves ProxGD and ProxSVRG/SAGA, and generalizes the results of SCSG in this case. Finally, we conduct several experiments and the experimental results are consistent with the theoretical results.

math.OC↗

arXiv2018

Theoretically motivated search and detection of non-thermal pulsations from PSRs J1747-2958, J2021+3651, and J1826-1256

Based on a theoretical selection of pulsars as candidates for detection at X-ray energies, we present an analysis of archival X-ray observations done with Chandra and XMM-Newton of PSR J1747-2958 (the pulsar in the "Mouse" nebula), PSR J2021+3651 (the pulsar in the "Dragonfly" nebula), and PSR J1826-1256. X-ray pulsations from PSR J1747-2958 and PSR J1826-1256 are detected for the first time, and a previously reported hint of an X-ray pulsation from PSR J2021+3651 is confirmed with a higher significance. We analyze these pulsars' spectra in regards to the theoretically predicted energy distribution, finding a remarkable agreement, and provide here a refined calculation of the model parameters taking into account the newly derived X-ray spectral data.

astro-ph.HE↗

arXiv2018

Optimal In-Place Suffix Sorting

The suffix array is a fundamental data structure for many applications that involve string searching and data compression. Designing time/space-efficient suffix array construction algorithms has attracted significant attention and considerable advances have been made for the past 20 years. We obtain the \emph{first} in-place suffix array construction algorithms that are optimal both in time and space for (read-only) integer alphabets. Concretely, we make the following contributions: 1. For integer alphabets, we obtain the first suffix sorting algorithm which takes linear time and uses only $O(1)$ workspace (the workspace is the total space needed beyond the input string and the output suffix array). The input string may be modified during the execution of the algorithm, but should be restored upon termination of the algorithm. 2. We strengthen the first result by providing the first in-place linear time algorithm for read-only integer alphabets with $|Σ|=O(n)$ (i.e., the input string cannot be modified). This algorithm settles the open problem posed by Franceschini and Muthukrishnan in ICALP 2007. The open problem asked to design in-place algorithms in $o(n\log n)$ time and ultimately, in $O(n)$ time for (read-only) integer alphabets with $|Σ| \leq n$. Our result is in fact slightly stronger since we allow $|Σ|=O(n)$. 3. Besides, for the read-only general alphabets (i.e., only comparisons are allowed), we present an optimal in-place $O(n\log n)$ time suffix sorting algorithm, recovering the result obtained by Franceschini and Muthukrishnan which was an open problem posed by Manzini and Ferragina in ESA 2002.

cs.DS↗

arXiv2018

Multi-Head Attention with Disagreement Regularization

Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions. In this work, we introduce a disagreement regularization to explicitly encourage the diversity among multiple attention heads. Specifically, we propose three types of disagreement regularization, which respectively encourage the subspace, the attended positions, and the output representation associated with each attention head to be different from other heads. Experimental results on widely-used WMT14 English-German and WMT17 Chinese-English translation tasks demonstrate the effectiveness and universality of the proposed approach.

cs.CL↗

arXiv2018

Visual Attention is Beyond One Single Saliency Map

Of later years, numerous bottom-up attention models have been proposed on different assumptions. However, the produced saliency maps may be different from each other even from the same input image. We also observe that human fixation map varies across time greatly. When people freely view an image, they tend to allocate attention at salient regions of large scale at first, and then search more and more detailed regions. In this paper, we argue that, for one input image visual attention cannot be described by only one single saliency map, and this mechanism should be modeled as a dynamic process. Under the frequency domain paradigm, we proposed a global inhibition model to mimic this process by suppressing the {\it non-saliency} in the input image; we also show that the dynamic process is influenced by one parameter in the frequency domain. Experiments illustrate that the proposed model is capable of predicting human dynamic fixation distribution.

cs.CV↗

arXiv2018

The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: a tomographic measurement of cosmic structure growth and expansion rate based on optimal redshift weights

We develop a new method, which is based on the optimal redshift weighting scheme, to extract the maximal tomographic information of baryonic acoustic oscillations (BAO) and redshift space distortions (RSD) from the extended Baryon Oscillation Spectroscopic Survey (eBOSS) Data Release 14 quasar (DR14Q) survey. We validate our method using the EZ mocks, and apply our pipeline to the eBOSS DR14Q sample in the redshift range of $0.8<z<2.2$. We report a joint measurement of $fσ_8$ and two-dimensional BAO parameters $D_{\rm A}$ and $H$ at four effective redshifts of $z_{\rm eff}=0.98, 1.23, 1.52$ and $1.94$, and provide the full data covariance matrix. Using our measurement combined with BOSS DR12, MGS and 6dFGS BAO measurements, we find that the existence of dark energy is supported by observations at a $7.4σ$ significance level. Combining our measurement with BOSS DR12 and Planck observations, we constrain the gravitational growth index to be $γ=0.580\pm0.082$, which is fully consistent with the prediction of general relativity. This paper is part of a set that analyses the eBOSS DR14 quasar sample.

astro-ph.CO↗

arXiv2018

Segmentation and Estimation of Change-point Models: False Positive Control and Confidence Regions

To segment a sequence of independent random variables at an unknown number of change-points, we introduce new procedures that are based on thresholding the likelihood ratio statistic. We also study confidence regions based on the likelihood ratio statistic for the change-points and joint confidence regions for the change-points and the parameter values. Applications to segment array CGH data are discussed.

math.ST↗

arXiv2018

Nuclear magnetic moments in covariant density functional theory

The nuclear magnetic moment is an important physical observable and serves as a useful tool for the stringent test of nuclear models. For the past decades, the covariant density functional theory and its extension have been proved to be successful in describing the nuclear ground-states and excited states properties. However, a long-standing problem is its failure to predict magnetic moments. This article reviews the recent progress in the description of the nuclear magnetic moments within the covariant density functional theory. In particular, the magnetic moments of spherical odd-A nuclei with doubly closed shell core plus or minus one nucleon and deformed odd-A nuclei.

nucl-th↗

arXiv2018

A Game-Theoretic Approach to Multi-Objective Resource Sharing and Allocation in Mobile Edge Clouds

Mobile edge computing seeks to provide resources to different delay-sensitive applications. However, allocating the limited edge resources to a number of applications is a challenging problem. To alleviate the resource scarcity problem, we propose sharing of resources among multiple edge computing service providers where each service provider has a particular utility to optimize. We model the resource allocation and sharing problem as a multi-objective optimization problem and present a \emph{Cooperative Game Theory} (CGT) based framework, where each edge service provider first satisfies its native applications and then shares its remaining resources (if available) with users of other providers. Furthermore, we propose an $\mathcal{O}(N)$ algorithm that provides allocation decisions from the \emph{core}, hence the obtained allocations are \emph{Pareto} optimal and the grand coalition of all the service providers is stable. Experimental results show that our proposed resource allocation and sharing framework improves the utility of all the service providers compared with the case where the service providers are working alone (no resource sharing). Our $\mathcal{O}(N)$ algorithm reduces the time complexity of obtaining a solution from the core by as much as 71.67\% when compared with the \emph{Shapley value}.

cs.GT↗

arXiv2018

$\varepsilon$-Coresets for Clustering (with Outliers) in Doubling Metrics

We study the problem of constructing $\varepsilon$-coresets for the $(k, z)$-clustering problem in a doubling metric $M(X, d)$. An $\varepsilon$-coreset is a weighted subset $S\subseteq X$ with weight function $w : S \rightarrow \mathbb{R}_{\geq 0}$, such that for any $k$-subset $C \in [X]^k$, it holds that $\sum_{x \in S}{w(x) \cdot d^z(x, C)} \in (1 \pm \varepsilon) \cdot \sum_{x \in X}{d^z(x, C)}$. We present an efficient algorithm that constructs an $\varepsilon$-coreset for the $(k, z)$-clustering problem in $M(X, d)$, where the size of the coreset only depends on the parameters $k, z, \varepsilon$ and the doubling dimension $\mathsf{ddim}(M)$. To the best of our knowledge, this is the first efficient $\varepsilon$-coreset construction of size independent of $|X|$ for general clustering problems in doubling metrics. To this end, we establish the first relation between the doubling dimension of $M(X, d)$ and the shattering dimension (or VC-dimension) of the range space induced by the distance $d$. Such a relation was not known before, since one can easily construct instances in which neither one can be bounded by (some function of) the other. Surprisingly, we show that if we allow a small $(1\pmε)$-distortion of the distance function $d$, and consider the notion of $τ$-error probabilistic shattering dimension, we can prove an upper bound of $O( \mathsf{ddim}(M)\cdot \log(1/\varepsilon) +\log\log{\frac{1}τ} )$ for the probabilistic shattering dimension for even weighted doubling metrics. We believe this new relation is of independent interest and may find other applications. We also study the robust coresets and centroid sets in doubling metrics. Our robust coreset construction leads to new results in clustering and property testing, and the centroid sets can be used to accelerate the local search algorithms for clustering problems.

cs.DS↗

arXiv2018

Odd Yao-Yao Graphs are Not Spanners

It is a long standing open problem whether Yao-Yao graphs $\mathsf{YY}_{k}$ are all spanners [li2002sparse]. Bauer and Damian [bauer2013infinite] showed that all $\mathsf{YY}_{6k}$ for $k \geq 6$ are spanners. Li and Zhan [li2016almost] generalized their result and proved that all even Yao-Yao graphs $\mathsf{YY}_{2k}$ are spanners (for $k\geq 42$). However, their technique cannot be extended to odd Yao-Yao graphs, and whether they are spanners are still elusive. In this paper, we show that, surprisingly, for any integer $k \geq 1$, there exist odd Yao-Yao graph $\mathsf{YY}_{2k+1}$ instances, which are not spanners.

cs.CG↗

arXiv2018

Network Classification in Temporal Networks Using Motifs

Network classification has a variety of applications, such as detecting communities within networks and finding similarities between those representing different aspects of the real world. However, most existing work in this area focus on examining static undirected networks without considering directed edges or temporality. In this paper, we propose a new methodology that utilizes feature representation for network classification based on the temporal motif distribution of the network and a null model for comparing against random graphs. Experimental results show that our method improves accuracy by up $10\%$ compared to the state-of-the-art embedding method in network classification, for tasks such as classifying network type, identifying communities in email exchange network, and identifying users given their app-switching behaviors.

cs.SI↗

arXiv2018

Combinatorial Multi-Armed Bandit with General Reward Functions

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the $\max()$ function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve $O(\log{T})$ distribution-dependent regret and $\tilde{O}(\sqrt{T})$ distribution-independent regret, where $T$ is the time horizon. We apply our results to the $K$-MAX problem and expected utility maximization problems. In particular, for $K$-MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the first $\tilde{O}(\sqrt T)$ bound on the $(1-ε)$-approximation regret of its online problem, for any $ε>0$.

cs.LG↗

arXiv2018

Giant Magnetoresistance in Hubbard Chains

We use numerically unbiased methods to show that the one-dimensional Hubbard model with periodically distributed on-site interactions already contains the minimal ingredients to display the phenomenon of magnetoresistance; i.e., by applying an external magnetic field, a dramatic enhancement on the charge transport is achieved. We reach this conclusion based on the computation of the Drude weight and of the single-particle density of states, applying twisted boundary condition averaging to reduce finite-size effects. The known picture that describes the giant magnetoresistance, by interpreting the scattering amplitudes of parallel or antiparallel polarized currents with local magnetizations, is obtained without having to resort to different entities; itinerant and localized charges are indistinguishable.

cond-mat.str-el↗

arXiv2018

Quickest Detection of Dynamic Events in Networks

The problem of quickest detection of dynamic events in networks is studied. At some unknown time, an event occurs, and a number of nodes in the network are affected by the event, in that they undergo a change in the statistics of their observations. It is assumed that the event is dynamic, in that it can propagate along the edges in the network, and affect more and more nodes with time. The event propagation dynamics is assumed to be unknown. The goal is to design a sequential algorithm that can detect a "significant" event, i.e., when the event has affected no fewer than $η$ nodes, as quickly as possible, while controlling the false alarm rate. Fully connected networks are studied first, and the results are then extended to arbitrarily connected networks. The designed algorithms are shown to be adaptive to the unknown propagation dynamics, and their first-order asymptotic optimality is demonstrated as the false alarm rate goes to zero. The algorithms can be implemented with linear computational complexity in the network size at each time step, which is critical for online implementation. Numerical simulations are provided to validate the theoretical results.

eess.SP↗

arXiv2018

An optimal nonconforming finite element method for the Stokes equations

In this paper, we propose and develop an optimal nonconforming finite element method for the Stokes equations approximated by the Crouzix-Raviart element for velocity and the continuous linear element for pressure. Previous result in using the stabilization method for this finite element pair is improved and then proven to be stable. Then, optimal order error estimate is obtained and numerical results show the accuracy and robustness of the method.

math.NA↗

arXiv2018

BRITS: Bidirectional Recurrent Imputation for Time Series

Time series are widely used as signals in many classification/regression tasks. It is ubiquitous that time series contains many missing values. Given multiple correlated time series data, how to fill in missing values and to predict their class labels? Existing imputation methods often impose strong assumptions of the underlying data generating process, such as linear dynamics in the state space. In this paper, we propose BRITS, a novel method based on recurrent neural networks for missing value imputation in time series data. Our proposed method directly learns the missing values in a bidirectional recurrent dynamical system, without any specific assumption. The imputed values are treated as variables of RNN graph and can be effectively updated during the backpropagation.BRITS has three advantages: (a) it can handle multiple correlated missing values in time series; (b) it generalizes to time series with nonlinear dynamics underlying; (c) it provides a data-driven imputation procedure and applies to general settings with missing data.We evaluate our model on three real-world datasets, including an air quality dataset, a health-care data, and a localization data for human activity. Experiments show that our model outperforms the state-of-the-art methods in both imputation and classification/regression accuracies.

cs.LG↗

arXiv2018

A PTAS for a Class of Stochastic Dynamic Programs

We develop a framework for obtaining polynomial time approximation schemes (PTAS) for a class of stochastic dynamic programs. Using our framework, we obtain the first PTAS for the following stochastic combinatorial optimization problems: \probemax: We are given a set of $n$ items, each item $i\in [n]$ has a value $X_i$ which is an independent random variable with a known (discrete) distribution $π_i$. We can {\em probe} a subset $P\subseteq [n]$ of items sequentially. Each time after {probing} an item $i$, we observe its value realization, which follows the distribution $π_i$. We can {\em adaptively} probe at most $m$ items and each item can be probed at most once. The reward is the maximum among the $m$ realized values. Our goal is to design an adaptive probing policy such that the expected value of the reward is maximized. To the best of our knowledge, the best known approximation ratio is $1-1/e$, due to Asadpour \etal~\cite{asadpour2015maximizing}. We also obtain PTAS for some generalizations and variants of the problem and some other problems.

cs.DS↗