arXiv subjects

Qiang Liu

Publications and source records attributed to Qiang Liu.

At least 451 records · Page 25Linked to original sources

arXiv2019

Learning Preferences and Demands in Visual Recommendation

Visual information is an important factor in recommender systems, in which users' selections consist of two components: \emph{preferences} and \emph{demands}. Some studies has been done for modeling users' preferences in visual recommendation. However, conventional methods models items in a common visual feature space, which may fail in capturing \emph{styles} of items. We propose a DeepStyle method for learning style features of items. DeepStyle eliminates the categorical information of items, which is dominant in the original visual feature space, based on a Convolutional Neural Networks (CNN) architecture. For modeling users' demands on different categories of items, the problem can be formulated as recommendation with contextual and sequential information. To solve this problem, we propose a Context-Aware Gated Recurrent Unit (CA-GRU) method, which can capture sequential and contextual information simultaneously. Furthermore, the aggregation of prediction on preferences and demands, i.e., prediction generated by DeepStyle and CA-GRU, can model users' selection behaviors more completely. Experiments conducted on real-world datasets illustrates the effectiveness of our proposed methods in visual recommendation.

cs.IR↗

arXiv2019

Stein Variational Gradient Descent With Matrix-Valued Kernels

Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference. In this work, we enhance SVGD by leveraging preconditioning matrices, such as the Hessian and Fisher information matrix, to incorporate geometric information into SVGD updates. We achieve this by presenting a generalization of SVGD that replaces the scalar-valued kernels in vanilla SVGD with more general matrix-valued kernels. This yields a significant extension of SVGD, and more importantly, allows us to flexibly incorporate various preconditioning matrices to accelerate the exploration in the probability landscape. Empirical results show that our method outperforms vanilla SVGD and a variety of baseline approaches over a range of real-world Bayesian inference tasks.

stat.ML↗

arXiv2019

Splitting Steepest Descent for Growing Neural Architectures

We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs. By leveraging a functional steepest descent idea, we derive a simple criterion for deciding the best subset of neurons to split and a splitting gradient for optimally updating the off-springs. Theoretically, our splitting strategy is a second-order functional steepest descent for escaping saddle points in an $\infty$-Wasserstein metric space, on which the standard parametric gradient descent is a first-order steepest descent. Our method provides a new computationally efficient approach for optimizing neural network structures, especially for learning lightweight neural architectures in resource-constrained settings.

cs.LG↗

arXiv2019

Learning-Assisted Secure End-to-End Network Slicing for Cyber-Physical Systems

There is a pressing need to interconnect physical systems such as power grid and vehicles for efficient management and safe operations. Owing to the diverse features of physical systems, there is hardly a one-size-fits-all networking solution for developing cyber-physical systems. Network slicing is a promising technology that allows network operators to create multiple virtual networks on top of a shared network infrastructure. These virtual networks can be tailored to meet the requirements of different cyber-physical systems. However, it is challenging to design secure network slicing solutions that can efficiently create end-to-end network slices for diverse cyber-physical systems. In this article, we discuss the challenges and security issues of network slicing, study learning-assisted network slicing solutions, and analyze their performance under the denial-of-service attack. We also present a design and implementation of a small-scale testbed for evaluating the network slicing solutions.

cs.NI↗

arXiv2019

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

We develop a Nonparametric Empirical Bayes (NEB) framework for compound estimation in the discrete linear exponential family, which includes a wide class of discrete distributions frequently arising from modern big data applications. We propose to directly estimate the Bayes shrinkage factor in the generalized Robbins' formula via solving a scalable convex program, which is carefully developed based on a RKHS representation of the Stein's discrepancy measure. The new NEB estimation framework is flexible for incorporating various structural constraints into the data driven rule, and provides a unified approach to compound estimation with both regular and scaled squared error losses. We develop theory to show that the class of NEB estimators enjoys strong asymptotic properties. Comprehensive simulation studies as well as analyses of real data examples are carried out to demonstrate the superiority of the NEB estimator over competing methods.

math.ST↗

arXiv2019

Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation

Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al. (2018a) proposed an approach that significantly reduces the variance of infinite-horizon off-policy evaluation by estimating the stationary density ratio, but at the cost of introducing potentially high biases due to the error in density ratio estimation. In this paper, we develop a bias-reduced augmentation of their method, which can take advantage of a learned value function to obtain higher accuracy. Our method is doubly robust in that the bias vanishes when either the density ratio or the value function estimation is perfect. In general, when either of them is accurate, the bias can also be reduced. Both theoretical and empirical results show that our method yields significant advantages over previous methods.

cs.LG↗

arXiv2019

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.

stat.ML↗

arXiv2019

Variational Inference with Tail-adaptive f-Divergence

Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importance sampling, which could have extremely large or infinite variances due to heavy tails of importance weights. In this paper, we propose a new class of tail-adaptive f-divergences that adaptively change the convex function f with the tail of the importance weights, in a way that theoretically guarantees finite moments, while simultaneously achieving mass-covering properties. We test our methods on Bayesian neural networks, as well as deep reinforcement learning in which our method is applied to improve a recent soft actor-critic (SAC) algorithm. Our results show that our approach yields significant advantages compared with existing methods based on classical KL and α-divergences.

cs.LG↗

arXiv2019

Improving Neural Language Modeling via Adversarial Training

Recently, substantial progress has been made in language modeling by using deep neural networks. However, in practice, large scale neural language models have been shown to be prone to overfitting. In this paper, we present a simple yet highly effective adversarial training mechanism for regularizing neural language models. The idea is to introduce adversarial noise to the output embedding layer while training the models. We show that the optimal adversarial noise yields a simple closed-form solution, thus allowing us to develop a simple and time efficient algorithm. Theoretically, we show that our adversarial mechanism effectively encourages the diversity of the embedding vectors, helping to increase the robustness of models. Empirically, we show that our method improves on the single model state-of-the-art results for language modeling on Penn Treebank (PTB) and Wikitext-2, achieving test perplexity scores of 46.01 and 38.07, respectively. When applied to machine translation, our method improves over various transformer-based translation baselines in BLEU scores on the WMT14 English-German and IWSLT14 German-English tasks.

cs.LG↗

arXiv2019

Polar Codes with Memory

Polar codes with memory (PCM) are proposed in this paper: a pair of consecutive code blocks containing a controlled number of mutual information bits. The shared mutual information bits of the succeeded block can help the failed block to recover. The underlying polar codes can employ any decoding scheme such as the successive cancellation (SC) decoding (PCM-SC), the belief propagation (BP) decoding (PCM-BP), and the successive cancellation list (SCL) decoding (PCM-SCL). The analysis shows that the packet error rate (PER) of PCM decreases to the order of PER squared while maintaining the same complexity as the underlying polar codes. Simulation results indicate that for PCM-SC, the PER is comparable to (less than 0.3 dB) the stand-alone SCL decoding with two lists for the block length $N=256$. The PER of PCM-SCL with $L$ lists can match that of the stand-alone SCL decoding with $2L$ lists. Two hardware decoders for PCM are also implemented: the in-serial (IS) decoder and the low-latency interleaved (LLI) decoder. For $N=256$, synthesis results show that in the worst case, the latency of the PCM LLI decoder is only $16.1\%$ of the adaptive SCL decoder with $L=2$, while the throughput is improved by 13 times compared to it.

cs.IT↗

arXiv2019

Learning Belief Representations for Imitation Learning in POMDPs

We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, and shown to be effective for reinforcement learning in POMDPs. In this work, we investigate the belief representation learning problem for generative adversarial imitation learning in POMDPs. Instead of training the belief module and the policy separately as suggested in prior work, we learn the belief module jointly with the policy, using a task-aware imitation loss to ensure that the representation is more aligned with the policy's objective. To improve robustness of representation, we introduce several informative belief regularization techniques, including multi-step prediction of dynamics and action-sequences. Evaluated on various partially observable continuous-control locomotion tasks, our belief-module imitation learning approach (BMIL) substantially outperforms several baselines, including the original GAIL algorithm and the task-agnostic belief learning algorithm. Extensive ablation analysis indicates the effectiveness of task-aware belief learning and belief regularization.

cs.LG↗

arXiv2019

Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning

Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, it is non-trivial to design such a reward function, and the only signal available is usually obtained at the end of a trajectory, also known as the episodic reward or return. In this work, we introduce a new algorithm for temporal credit assignment, which learns to decompose the episodic return back to each time-step in the trajectory using deep neural networks. With this learned reward signal, the learning efficiency can be substantially improved for episodic reinforcement learning. In particular, we find that expressive language models such as the Transformer can be adopted for learning the importance and the dependency of states in the trajectory, therefore providing high-quality and interpretable learned reward signals. We have performed extensive experiments on a set of MuJoCo continuous locomotive control tasks with only episodic returns and demonstrated the effectiveness of our algorithm.

cs.LG↗

arXiv2019

PI-BA Bundle Adjustment Acceleration on Embedded FPGAs with Co-observation Optimization

Bundle adjustment (BA) is a fundamental optimization technique used in many crucial applications, including 3D scene reconstruction, robotic localization, camera calibration, autonomous driving, space exploration, street view map generation etc. Essentially, BA is a joint non-linear optimization problem, and one which can consume a significant amount of time and power, especially for large optimization problems. Previous approaches of optimizing BA performance heavily rely on parallel processing or distributed computing, which trade higher power consumption for higher performance. In this paper we propose π-BA, the first hardware-software co-designed BA engine on an embedded FPGA-SoC that exploits custom hardware for higher performance and power efficiency. Specifically, based on our key observation that not all points appear on all images in a BA problem, we designed and implemented a Co-Observation Optimization technique to accelerate BA operations with optimized usage of memory and computation resources. Experimental results confirm that π-BA outperforms the existing software implementations in terms of performance and power consumption.

eess.IV↗

arXiv2019

Pulse strategy for suppressing spreading on networks

In networked spreading models, each node can infect its neighbors and cure spontaneously. The curing is assumed to occur uniformly over time. A pulse immunization/curing strategy is more efficient and broadly applied to suppressing spreading processes. We model the epidemic process by the basic Susceptible-Infected (SI) process with a pulse curing and incorporate the underlying contact network. The mean-field epidemic threshold of the pulse SI model is shown to be $\frac{1}{λ_1}\ln\frac{1}{1-p}$, where $λ_1$ and $p$ are the largest eigenvalue of the adjacency matrix and the fraction of nodes covered by each curing, respectively. Compared to the extensively studied uniform curing process, we show that the pulse curing strategy saves about $36.8$\%, i.e. $p\approx 0.632$, of the number of curing operations invariant to the network structure. Our results may help related policy makers to estimate the cost of controlling spreading processes.

physics.soc-ph↗

arXiv2019

Acoustic-optic Q-switched cavityless weak-feedback laser based on Nd:GdVO4 bounce geometry

An AOQ (acoustic-optic Q-switched) laser with cavityless weak-feedback configuration is demonstrated based on a Nd:GdVO4 bounce geometry. The laser can operate at a repetition up to 500 kHz with the output power above 4W. The pulse-width at 100 kHz reaches 5.2 ns, which is 4.3 times the round-trip time. A theory of Q-switched cavityless weak-feedback laser is proposed for the first time, to our best knowledge. This theory is very suitable for analyzing short pulses comparable to round-trip time. By utilizing the theory, simulation is implemented for our experimental conditions. The consistence between the simulation and the experimental results proves the validity of our theory.

physics.optics↗

arXiv2019

Short pulse close to round-trip time generated by cavityless high gain Nd:GdVO4 bounce geometry

In this paper, laser pulses with pulse-widths approach to the round-trip time are generated by utilizing a cavityless high gain Nd:GdVO4 bounce geometry. By adopting an EOQ (electro-optics Q-switch), pulse-widths of 1.36 ns, 1.82 ns, and 2.39 ns are achieved at three effective cavity lengths respectively. All these pulse-widths are close to the round-trip time of corresponding effective cavity lengths. Moreover, the output power reaches watt-level and the repetition rate is kHz-level, meanwhile the M2 factor is less than 1.3. Spectrally, the laser has a continuous spectrum with 10 dB linewidth of 0.2 nm.

physics.optics↗

arXiv2019

Learning Self-Imitating Diverse Policies

The success of popular algorithms for deep reinforcement learning, such as policy-gradients and Q-learning, relies heavily on the availability of an informative reward signal at each timestep of the sequential decision-making process. When rewards are only sparsely available during an episode, or a rewarding feedback is provided only after episode termination, these algorithms perform sub-optimally due to the difficultly in credit assignment. Alternatively, trajectory-based policy optimization methods, such as cross-entropy method and evolution strategies, do not require per-timestep rewards, but have been found to suffer from high sample complexity by completing forgoing the temporal nature of the problem. Improving the efficiency of RL algorithms in real-world problems with sparse or episodic rewards is therefore a pressing need. In this work, we introduce a self-imitation learning algorithm that exploits and explores well in the sparse and episodic reward settings. We view each policy as a state-action visitation distribution and formulate policy optimization as a divergence minimization problem. We show that with Jensen-Shannon divergence, this divergence minimization problem can be reduced into a policy-gradient algorithm with shaped rewards learned from experience replays. Experimental results indicate that our algorithm works comparable to existing algorithms in environments with dense rewards, and significantly better in environments with sparse and episodic rewards. We then discuss limitations of self-imitation learning, and propose to solve them by using Stein variational policy gradient descent with the Jensen-Shannon kernel to learn multiple diverse policies. We demonstrate its effectiveness on a challenging variant of continuous-control MuJoCo locomotion tasks.

stat.ML↗

arXiv2019

Exoplanets in the Antarctic sky. II. 116 Transiting Exoplanet Candidates Found by AST3-II (CHESPA) within the Southern CVZ of TESS

We report first results from the CHinese Exoplanet Searching Program from Antarctica (CHESPA)---a wide-field high-resolution photometric survey for transiting exoplanets carried out using telescopes of the AST3 (Antarctic Survey Telescopes times 3) project. There are now three telescopes (AST3-I, AST3-II, and CSTAR-II) operating at Dome A---the highest point on the Antarctic Plateau---in a fully automatic and remote mode to exploit the superb observing conditions of the site, and its long and uninterrupted polar nights. The search for transiting exoplanets is one of the key projects for AST3. During the Austral winters of 2016 and 2017 we used the AST3-II telescope to survey a set of target fields near the southern ecliptic pole, falling within the continuous viewing zone of the TESS mission \citep{Ricker10}. The first data release of the 2016 data, including images, catalogs and lightcurves of 26578 bright stars ($7.5\le i \le15$) was presented in \citet{Zhang18}. The best precision, as measured by the RMS of the lightcurves at the optimum magnitude of the survey ($i=10$), is around 2\,mmag. We detect 222 objects with plausible transit signals from these data, 116 of which are plausible transiting exoplanet candidates according to their stellar properties as given by the TESS Input Catalog \citep{Stassun17}, Gaia DR2 \citep{Gaia18} and TESS-HERMES spectroscopy \citep{Sharma18}. With the first data release from TESS expected in late 2018, this candidate list will be a timely for improving the rejection of potential false positives.

astro-ph.EP↗