arXiv subjects

Yue Wang

Publications and source records attributed to Yue Wang.

At least 883 records · Page 49Linked to original sources

arXiv2018

Finite Sample Analysis of the GTD Policy Evaluation Algorithms in Markov Setting

In reinforcement learning (RL) , one of the key components is policy evaluation, which aims to estimate the value function (i.e., expected long-term accumulated reward) of a policy. With a good policy evaluation method, the RL algorithms will estimate the value function more accurately and find a better policy. When the state space is large or continuous \emph{Gradient-based Temporal Difference(GTD)} policy evaluation algorithms with linear function approximation are widely used. Considering that the collection of the evaluation data is both time and reward consuming, a clear understanding of the finite sample performance of the policy evaluation algorithms is very important to reinforcement learning. Under the assumption that data are i.i.d. generated, previous work provided the finite sample analysis of the GTD algorithms with constant step size by converting them into convex-concave saddle point problems. However, it is well-known that, the data are generated from Markov processes rather than i.i.d. in RL problems.. In this paper, in the realistic Markov setting, we derive the finite sample bounds for the general convex-concave saddle point problems, and hence for the GTD algorithms. We have the following discussions based on our bounds. (1) With variants of step size, GTD algorithms converge. (2) The convergence rate is determined by the step size, with the mixing time of the Markov process as the coefficient. The faster the Markov processes mix, the faster the convergence. (3) We explain that the experience replay trick is effective by improving the mixing property of the Markov process. To the best of our knowledge, our analysis is the first to provide finite sample bounds for the GTD algorithms in Markov setting.

cs.LG↗

arXiv2018

Deep Co-investment Network Learning for Financial Assets

Most recent works model the market structure of the stock market as a correlation network of the stocks. They apply pre-defined patterns to extract correlation information from the time series of stocks. Without considering the influences of the evolving market structure to the market index trends, these methods hardly obtain the market structure models which are compatible with the market principles. Advancements in deep learning have shown their incredible modeling capacity on various finance-related tasks. However, the learned inner parameters, which capture the essence of the finance time series, are not further exploited about their representation in the financial fields. In this work, we model the financial market structure as a deep co-investment network and propose a Deep Co-investment Network Learning (DeepCNL) method. DeepCNL automatically learns deep co-investment patterns between any pairwise stocks, where the rise-fall trends of the market index are used for distance supervision. The learned inner parameters of the trained DeepCNL, which encodes the temporal dynamics of deep co-investment patterns, are used to build the co-investment network between the stocks as the investment structure of the corresponding market. We verify the effectiveness of DeepCNL on the real-world stock data and compare it with the existing methods on several financial tasks. The experimental results show that DeepCNL not only has the ability to better reflect the stock market structure that is consistent with widely-acknowledged financial principles but also is more capable to approximate the investment activities which lead to the stock performance reported in the real news or research reports than other alternatives.

cs.CE↗

arXiv2018

Identifying The Most Informative Features Using A Structurally Interacting Elastic Net

Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information loss. To address this problem, we propose a new graph-based structurally interacting elastic net method for feature selection. Specifically, we commence by constructing feature graphs that can incorporate pairwise relationship between samples. With the feature graphs to hand, we propose a new information theoretic criterion to measure the joint relevance of different pairwise feature combinations with respect to the target feature graph representation. This measure is used to obtain a structural interaction matrix where the elements represent the proposed information theoretic measure between feature pairs. We then formulate a new optimization model through the combination of the structural interaction matrix and an elastic net regression model for the feature subset selection problem. This allows us to a) preserve the information of the original vectorial space, b) remedy the information loss of the original feature space caused by using graph representation, and c) promote a sparse solution and also encourage correlated features to be selected. Because the proposed optimization problem is non-convex, we develop an efficient alternating direction multiplier method (ADMM) to locate the optimal solutions. Extensive experiments on various datasets demonstrate the effectiveness of the proposed methods.

cs.LG↗

arXiv2018

Trust-based Multi-Robot Symbolic Motion Planning with a Human-in-the-Loop

Symbolic motion planning for robots is the process of specifying and planning robot tasks in a discrete space, then carrying them out in a continuous space in a manner that preserves the discrete-level task specifications. Despite progress in symbolic motion planning, many challenges remain, including addressing scalability for multi-robot systems and improving solutions by incorporating human intelligence. In this paper, distributed symbolic motion planning for multi-robot systems is developed to address scalability. More specifically, compositional reasoning approaches are developed to decompose the global planning problem, and atomic propositions for observation, communication, and control are proposed to address inter-robot collision avoidance. To improve solution quality and adaptability, a dynamic, quantitative, and probabilistic human-to-robot trust model is developed to aid this decomposition. Furthermore, a trust-based real-time switching framework is proposed to switch between autonomous and manual motion planning for tradeoffs between task safety and efficiency. Deadlock- and livelock-free algorithms are designed to guarantee reachability of goals with a human-in-the-loop. A set of non-trivial multi-robot simulations with direct human input and trust evaluation are provided demonstrating the successful implementation of the trust-based multi-robot symbolic motion planning methods.

cs.RO↗

arXiv2018

Low-complexity optimization for Two-Dimensional Direction-of-arrival Estimation via Decoupled Atomic Norm Minimization

This paper presents an efficient optimization technique for super-resolution two-dimensional (2D) direction of arrival (DOA) estimation by introducing a new formulation of atomic norm minimization (ANM). ANM allows gridless angle estimation for correlated sources even when the number of snapshots is far less than the antenna size, yet it incurs huge computational cost in 2D processing. This paper introduces a novel formulation of ANM via semi-definite programming, which expresses the original high-dimensional problem by two decoupled Toeplitz matrices in one dimension, followed by 1D angle estimation with automatic angle pairing. Compared with the state-of-the-art 2D ANM, the proposed technique reduces the computational complexity by several orders of magnitude with respect to the antenna size, while retaining the benefits of ANM in terms of super-resolution performance with use of a small number of measurements, and robustness to source correlation and noise. The complexity benefits are particularly attractive for large-scale antenna systems such as massive MIMO and radio astronomy.

eess.SP↗

arXiv2018

Multi-session Map Construction in Outdoor Dynamic Environment

Map construction in large scale outdoor environment is of importance for robots to robustly fulfill their tasks. Massive sessions of data should be merged to distinguish low dynamics in the map, which otherwise might debase the performance of localization and navigation algorithms. In this paper we propose a method for multi-session map construction in large scale outdoor environment using 3D LiDAR. To efficiently align the maps from different sessions, a laser-based loop closure detection method is integrated and the sequential information within the submaps is utilized for higher robustness. Furthermore, a dynamic detection method is proposed to detect dynamics in the overlapping areas among sessions of maps. We test the method in the real-world environment with a VLP-16 Velodyne LiDAR and the experimental results prove the validity and robustness of the proposed method.

cs.RO↗

arXiv2018

The viscosity of charged particles in the weakly ionized plasma with power-law distributions

We study the viscosity of light charged particles in weakly ionized plasma with the power-law q-distributions by using the Boltzmann equation of transport and the motion equation of hydrodynamics. The nonequilibrium plasma is considered to be space inhomogeneous and without magnetic field. We derive the expressions of the viscosity coefficients of the electrons and the ions in the q-distributed plasma, including the first and the second viscosity coefficients respectively. It is shown that these new viscosity coefficients depend strongly on the q-parameters, and when we take the limit of q to 1, they perfectly return to those in the plasma with a Maxwellian distribution. The discussions presented in this paper can also be applied to the plasmas with the kappa-distributions.

physics.plasm-ph↗

arXiv2018

LocNet: Global localization in 3D point clouds for mobile vehicles

Global localization in 3D point clouds is a challenging problem of estimating the pose of vehicles without any prior knowledge. In this paper, a solution to this problem is presented by achieving place recognition and metric pose estimation in the global prior map. Specifically, we present a semi-handcrafted representation learning method for LiDAR point clouds using siamese LocNets, which states the place recognition problem to a similarity modeling problem. With the final learned representations by LocNet, a global localization framework with range-only observations is proposed. To demonstrate the performance and effectiveness of our global localization system, KITTI dataset is employed for comparison with other algorithms, and also on our long-time multi-session datasets for evaluation. The result shows that our system can achieve high accuracy.

cs.RO↗

arXiv2018

Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions

The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e.g., GoogLeNet, ResNet and Wide ResNet). Further, the high energy consumption of convolutions limits its deployment on mobile devices. To this end, we proposed a simple yet effective scheme for compressing convolutions though applying k-means clustering on the weights, compression is achieved through weight-sharing, by only recording $K$ cluster centers and weight assignment indexes. We then introduced a novel spectrally relaxed $k$-means regularization, which tends to make hard assignments of convolutional layer weights to $K$ learned cluster centers during re-training. We additionally propose an improved set of metrics to estimate energy consumption of CNN hardware implementations, whose estimation results are verified to be consistent with previously proposed energy estimation tool extrapolated from actual hardware measurements. We finally evaluated Deep $k$-Means across several CNN models in terms of both compression ratio and energy consumption reduction, observing promising results without incurring accuracy loss. The code is available at https://github.com/Sandbox3aster/Deep-K-Means

cs.LG↗

arXiv2018

Learning-to-Ask: Knowledge Acquisition via 20 Questions

Almost all the knowledge empowered applications rely upon accurate knowledge, which has to be either collected manually with high cost, or extracted automatically with unignorable errors. In this paper, we study 20 Questions, an online interactive game where each question-response pair corresponds to a fact of the target entity, to acquire highly accurate knowledge effectively with nearly zero labor cost. Knowledge acquisition via 20 Questions predominantly presents two challenges to the intelligent agent playing games with human players. The first one is to seek enough information and identify the target entity with as few questions as possible, while the second one is to leverage the remaining questioning opportunities to acquire valuable knowledge effectively, both of which count on good questioning strategies. To address these challenges, we propose the Learning-to-Ask (LA) framework, within which the agent learns smart questioning strategies for information seeking and knowledge acquisition by means of deep reinforcement learning and generalized matrix factorization respectively. In addition, a Bayesian approach to represent knowledge is adopted to ensure robustness to noisy user responses. Simulating experiments on real data show that LA is able to equip the agent with effective questioning strategies, which result in high winning rates and rapid knowledge acquisition. Moreover, the questioning strategies for information seeking and knowledge acquisition boost the performance of each other, allowing the agent to start with a relatively small knowledge set and quickly improve its knowledge base in the absence of constant human supervision.

cs.AI↗

arXiv2018

Maximum Volume Inscribed Ellipsoid: A New Simplex-Structured Matrix Factorization Framework via Facet Enumeration and Convex Optimization

Consider a structured matrix factorization model where one factor is restricted to have its columns lying in the unit simplex. This simplex-structured matrix factorization (SSMF) model and the associated factorization techniques have spurred much interest in research topics over different areas, such as hyperspectral unmixing in remote sensing, topic discovery in machine learning, to name a few. In this paper we develop a new theoretical SSMF framework whose idea is to study a maximum volume ellipsoid inscribed in the convex hull of the data points. This maximum volume inscribed ellipsoid (MVIE) idea has not been attempted in prior literature, and we show a sufficient condition under which the MVIE framework guarantees exact recovery of the factors. The sufficient recovery condition we show for MVIE is much more relaxed than that of separable non-negative matrix factorization (or pure-pixel search); coincidentally it is also identical to that of minimum volume enclosing simplex, which is known to be a powerful SSMF framework for non-separable problem instances. We also show that MVIE can be practically implemented by performing facet enumeration and then by solving a convex optimization problem. The potential of the MVIE framework is illustrated by numerical results.

stat.ML↗

arXiv2018

Topological and Algebraic Properties of Chernoff Information between Gaussian Graphs

In this paper, we want to find out the determining factors of Chernoff information in distinguishing a set of Gaussian graphs. We find that Chernoff information of two Gaussian graphs can be determined by the generalized eigenvalues of their covariance matrices. We find that the unit generalized eigenvalue doesn't affect Chernoff information and its corresponding dimension doesn't provide information for classification purpose. In addition, we can provide a partial ordering using Chernoff information between a series of Gaussian trees connected by independent grafting operations. With the relationship between generalized eigenvalues and Chernoff information, we can do optimal linear dimension reduction with least loss of information for classification.

cs.IT↗

arXiv2018

Code Completion with Neural Attention and Pointer Networks

Intelligent code completion has become an essential research task to accelerate modern software development. To facilitate effective code completion for dynamically-typed programming languages, we apply neural language models by learning from large codebases, and develop a tailored attention mechanism for code completion. However, standard neural language models even with attention mechanism cannot correctly predict the out-of-vocabulary (OoV) words that restrict the code completion performance. In this paper, inspired by the prevalence of locally repeated terms in program source code, and the recently proposed pointer copy mechanism, we propose a pointer mixture network for better predicting OoV words in code completion. Based on the context, the pointer mixture network learns to either generate a within-vocabulary word through an RNN component, or regenerate an OoV word from local context through a pointer component. Experiments on two benchmarked datasets demonstrate the effectiveness of our attention mechanism and pointer mixture network on the code completion task.

cs.CL↗

arXiv2018

Bounded Policy Synthesis for POMDPs with Safe-Reachability Objectives

Planning robust executions under uncertainty is a fundamental challenge for building autonomous robots. Partially Observable Markov Decision Processes (POMDPs) provide a standard framework for modeling uncertainty in many applications. In this work, we study POMDPs with safe-reachability objectives, which require that with a probability above some threshold, a goal state is eventually reached while keeping the probability of visiting unsafe states below some threshold. This POMDP formulation is different from the traditional POMDP models with optimality objectives and we show that in some cases, POMDPs with safe-reachability objectives can provide a better guarantee of both safety and reachability than the existing POMDP models through an example. A key algorithmic problem for POMDPs is policy synthesis, which requires reasoning over a vast space of beliefs (probability distributions). To address this challenge, we introduce the notion of a goal-constrained belief space, which only contains beliefs reachable from the initial belief under desired executions that can achieve the given safe-reachability objective. Our method compactly represents this space over a bounded horizon using symbolic constraints, and employs an incremental Satisfiability Modulo Theories (SMT) solver to efficiently search for a valid policy over it. We evaluate our method using a case study involving a partially observable robotic domain with uncertain obstacles. The results show that our method can synthesize policies over large belief spaces with a small number of SMT solver calls by focusing on the goal-constrained belief space.

cs.RO↗

arXiv2018

Regularity of Solutions of the Camassa-Holm Equations with Fractional Laplacian Viscosity

We study the existence, uniqueness and regularity of solutions to the $n$-dimensional ($n=2,3$) Camassa-Holm equations with fractional Laplacian viscosity with smooth initial data. It is a coupled system between the Navier-Stokes equations with nonlocal viscosity and a Helmholtz equation. The main difficulty lies in establishing some a priori estimates for the fractional Laplacian viscosity. To achieve this, we need to explore suitable fractional-power Sobolev-type estimates, and bilinear estimates for fractional derivatives. Especially, for the critical case $\displaystyle s=\frac{n}{4}$ with $n=2,3$, we will make extra efforts for acquiring the expected estimates as obtained in the case $\displaystyle \frac{n}{4}<s<1$. By the aid of the fractional Leibniz rule and the nonlocal version of Ladyzhenskaya's inequality, we prove the existence, uniqueness and regularity to the Camassa-Holm equations under study by the energy method and a bootstrap argument, which rely crucially on the fractional Laplacian viscosity. In particular, under the critical case $s=\dfrac{n}{4}$, the nonlocal version of Ladyzhenskaya's inequality is skillfully used, and the smallness of initial data in several Sobolev spaces is required to gain the desired results concernig existence, uniqueness and regularity.

math.AP↗

arXiv2018

Im2Avatar: Colorful 3D Reconstruction from a Single Image

Existing works on single-image 3D reconstruction mainly focus on shape recovery. In this work, we study a new problem, that is, simultaneously recovering 3D shape and surface color from a single image, namely "colorful 3D reconstruction". This problem is both challenging and intriguing because the ability to infer textured 3D model from a single image is at the core of visual understanding. Here, we propose an end-to-end trainable framework, Colorful Voxel Network (CVN), to tackle this problem. Conditioned on a single 2D input, CVN learns to decompose shape and surface color information of a 3D object into a 3D shape branch and a surface color branch, respectively. Specifically, for the shape recovery, we generate a shape volume with the state of its voxels indicating occupancy. For the surface color recovery, we combine the strength of appearance hallucination and geometric projection by concurrently learning a regressed color volume and a 2D-to-3D flow volume, which are then fused into a blended color volume. The final textured 3D model is obtained by sampling color from the blended color volume at the positions of occupied voxels in the shape volume. To handle the severe sparse volume representations, a novel loss function, Mean Squared False Cross-Entropy Loss (MSFCEL), is designed. Extensive experiments demonstrate that our approach achieves significant improvement over baselines, and shows great generalization across diverse object categories and arbitrary viewpoints.

cs.CV↗

arXiv2018

Differentially Private Confidence Intervals for Empirical Risk Minimization

The process of data mining with differential privacy produces results that are affected by two types of noise: sampling noise due to data collection and privacy noise that is designed to prevent the reconstruction of sensitive information. In this paper, we consider the problem of designing confidence intervals for the parameters of a variety of differentially private machine learning models. The algorithms can provide confidence intervals that satisfy differential privacy (as well as the more recently proposed concentrated differential privacy) and can be used with existing differentially private mechanisms that train models using objective perturbation and output perturbation.

cs.LG↗

arXiv2018

Some insights into the behaviour of millimetre wave spectrum on key 5G cellular KPIs

This invited paper discusses the challenging task of assessing the millmeter-wave spectrum for the use of 5G mobile systems. 3 key KPIs related to mobile systems, coverage, capacity and mobility are individually assessed and quantified with a simplistic figure of merit (FoM). The coverage and capacity KPIs are assessed through simulations while an analytical approach is developed for mobility. The key message is that the FoM degrades with the higher as the frequencies increase in this range. This indicates that there will be more challenges in deploying millimeter wave systems in higher frequencies, but these are not insurmountable. Many of these challenges are addressed in the mmMAGIC project and possible solutions are derived.

cs.IT↗