arXiv subjects

Liang Zhao

Publications and source records attributed to Liang Zhao.

At least 469 records · Page 26Linked to original sources

arXiv2018

Occlusion Aware Unsupervised Learning of Optical Flow

It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning.

cs.CV↗

arXiv2018

MIS-SLAM: Real-time Large Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing

Real-time simultaneously localization and dense mapping is very helpful for providing Virtual Reality and Augmented Reality for surgeons or even surgical robots. In this paper, we propose MIS-SLAM: a complete real-time large scale dense deformable SLAM system with stereoscope in Minimal Invasive Surgery based on heterogeneous computing by making full use of CPU and GPU. Idled CPU is used to perform ORB- SLAM for providing robust global pose. Strategies are taken to integrate modules from CPU and GPU. We solved the key problem raised in previous work, that is, fast movement of scope and blurry images make the scope tracking fail. Benefiting from improved localization, MIS-SLAM can achieve large scale scope localizing and dense mapping in real-time. It transforms and deforms current model and incrementally fuses new observation while keeping vivid texture. In-vivo experiments conducted on publicly available datasets presented in the form of videos demonstrate the feasibility and practicality of MIS-SLAM for potential clinical purpose.

cs.CV↗

arXiv2018

Learning to Explore with Meta-Policy Gradient

The performance of off-policy learning, including deep Q-learning and deep deterministic policy gradient (DDPG), critically depends on the choice of the exploration policy. Existing exploration methods are mostly based on adding noise to the on-going actor policy and can only explore \emph{local} regions close to what the actor policy dictates. In this work, we develop a simple meta-policy gradient algorithm that allows us to adaptively learn the exploration policy in DDPG. Our algorithm allows us to train flexible exploration behaviors that are independent of the actor policy, yielding a \emph{global exploration} that significantly speeds up the learning process. With an extensive study, we show that our method significantly improves the sample-efficiency of DDPG on a variety of reinforcement learning tasks.

cs.LG↗

arXiv2018

Superfast CUR Matrix Algorithms, Their Pre-Processing and Extensions

We study superfast algorithms that computes low rank approximation of a matrix (hereafter referred to as LRA) that use much fewer memory cells and arithmetic operations than the input matrix has entries. We first specify a family of 2mn matrices of size m*n such that for almost 50% of them any superfast LRA algorithm fails to improve the poor trivial approximation by the matrix filled with zeros, but then we prove that the class of all such hard inputs is narrow - the cross-approximation (hereafter {C-A}) superfast iterations as well as some more primitive superfast algorithms compute reasonably accurate LRAs in their transparent CUR form (i) to any matrix allowing close LRA except for small norm perturbations of matrices of an algebraic variety of a smaller dimension, (ii) to the average matrix allowing close LRA, (iii) to the average sparse matrix allowing close LRA and (iv) with a high probability to any matrix allowing close LRA if it is pre-processed fast with a random Gaussian, SRHT or SRFT multiplier. Moreover empirically the output LRAs remain accurate when we perform the computations superfast by replacing such a multiplier with one of our sparse and structured multipliers. Our techniques, auxiliary results and extensions may be of some independent interest. We analyze C-A and other superfast algorithms twice -- based on two well-known sufficient criteria for obtaining accurate LRAs. We provide a distinct proof in the case of superfast variant of randomized algorithms of [DMM08], improve a decade-old estimate for the norm of the inverse of a Gaussian matrix, prove such an estimate also in the case of a sparse Gaussian matrix, present some novel advanced pre-processing techniques for fast and superfast computation of LRA, and extend our results to dramatic acceleration of the Fast Multipole Method (FMM) and the Conjugate Gradient algorithms.

math.NA↗

arXiv2018

Network Unfolding Map by Edge Dynamics Modeling

The emergence of collective dynamics in neural networks is a mechanism of the animal and human brain for information processing. In this paper, we develop a computational technique using distributed processing elements in a complex network, which are called particles, to solve semi-supervised learning problems. Three actions govern the particles' dynamics: generation, walking, and absorption. Labeled vertices generate new particles that compete against rival particles for edge domination. Active particles randomly walk in the network until they are absorbed by either a rival vertex or an edge currently dominated by rival particles. The result from the model evolution consists of sets of edges arranged by the label dominance. Each set tends to form a connected subnetwork to represent a data class. Although the intrinsic dynamics of the model is a stochastic one, we prove there exists a deterministic version with largely reduced computational complexity; specifically, with linear growth. Furthermore, the edge domination process corresponds to an unfolding map in such way that edges "stretch" and "shrink" according to the vertex-edge dynamics. Consequently, the unfolding effect summarizes the relevant relationships between vertices and the uncovered data classes. The proposed model captures important details of connectivity patterns over the vertex-edge dynamics evolution, in contrast to previous approaches which focused on only vertex or only edge dynamics. Computer simulations reveal that the new model can identify nonlinear features in both real and artificial data, including boundaries between distinct classes and overlapping structures of data.

cs.AI↗

arXiv2018

Network community detection via iterative edge removal in a flocking-like system

We present a network community-detection technique based on properties that emerge from a nature-inspired system of aligning particles. Initially, each vertex is assigned a random-direction unit vector. A nonlinear dynamic law is established so that neighboring vertices try to become aligned with each other. After some time, the system stops and edges that connect the least-aligned pairs of vertices are removed. Then the evolution starts over without the removed edges, and after enough number of removal rounds, each community becomes a connected component. The proposed approach is evaluated using widely-accepted benchmarks and real-world networks. Experimental results reveal that the method is robust and excels on a wide variety of networks. Moreover, for large sparse networks, the edge-removal process runs in quasilinear time, which enables application in large-scale networks.

cs.SI↗

arXiv2018

Water-phospholipid interactions at the interface of lipid membranes: comparison of different force fields

Water-phospholipid interactions at the lipid bilayer/water interfaces are of essential importance for the dynamics, stability and function of biological membrane, and are also strongly associated with numerous biological processes at the interfaces of lipid bilayers. Various force fields, such as the united-atom Berger force field, its two improved versions by Kukol and by Poger, and the all-atom Slipid force field developed recently, can be applied to simulating the structures of lipid bilayer, with their structural predictions in good agreement with experimental data. In this work, we show that despite the similarity in structural predictions of lipid bilayers, there are observable differences in formation of hydrogen bonds and the interaction energy profiles between water and phospholipid groups at the lipid bilayer/water interfaces, when four force fields for dipalmitoylphosphatidylcholine (DPPC) phospholipids are employed in molecular dynamics simulations. In particular, the Slipid force field yields more hydrogen bonds between water and phospholipids and more symmetrical interaction energy distributions for the two carboxylic groups on their respective acyl tails, compared to the Berger and its two improved force fields. These differences are mainly attributed to the different interfacial water distributions and ability to form hydrogen bonds between interfacial water and oxygen atoms of the DPPC lipids using different force fields. These results would be helpful in understanding the behaviors of water as well as its interaction with phospholipids at the lipid bilayer/water interfaces, and provide a guide for making the appropriate choice on the force field in simulations of lipid bilayers.

cond-mat.soft↗

arXiv2017

Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency

Learning to reconstruct depths in a single image by watching unlabeled videos via deep convolutional network (DCN) is attracting significant attention in recent years. In this paper, we introduce a surface normal representation for unsupervised depth estimation framework. Our estimated depths are constrained to be compatible with predicted normals, yielding more robust geometry results. Specifically, we formulate an edge-aware depth-normal consistency term, and solve it by constructing a depth-to-normal layer and a normal-to-depth layer inside of the DCN. The depth-to-normal layer takes estimated depths as input, and computes normal directions using cross production based on neighboring pixels. Then given the estimated normals, the normal-to-depth layer outputs a regularized depth map through local planar smoothness. Both layers are computed with awareness of edges inside the image to help address the issue of depth/normal discontinuity and preserve sharp edges. Finally, to train the network, we apply the photometric error and gradient smoothness for both depth and normal predictions. We conducted experiments on both outdoor (KITTI) and indoor (NYUv2) datasets, and show that our algorithm vastly outperforms state of the art, which demonstrates the benefits from our approach.

cs.CV↗

arXiv2017

Feature learning in feature-sample networks using multi-objective optimization

Data and knowledge representation are fundamental concepts in machine learning. The quality of the representation impacts the performance of the learning model directly. Feature learning transforms or enhances raw data to structures that are effectively exploited by those models. In recent years, several works have been using complex networks for data representation and analysis. However, no feature learning method has been proposed for such category of techniques. Here, we present an unsupervised feature learning mechanism that works on datasets with binary features. First, the dataset is mapped into a feature--sample network. Then, a multi-objective optimization process selects a set of new vertices to produce an enhanced version of the network. The new features depend on a nonlinear function of a combination of preexisting features. Effectively, the process projects the input data into a higher-dimensional space. To solve the optimization problem, we design two metaheuristics based on the lexicographic genetic algorithm and the improved strength Pareto evolutionary algorithm (SPEA2). We show that the enhanced network contains more information and can be exploited to improve the performance of machine learning methods. The advantages and disadvantages of each optimization strategy are discussed.

cs.AI↗

arXiv2017

Online and Distributed Robust Regressions under Adversarial Data Corruption

In today's era of big data, robust least-squares regression becomes a more challenging problem when considering the adversarial corruption along with explosive growth of datasets. Traditional robust methods can handle the noise but suffer from several challenges when applied in huge dataset including 1) computational infeasibility of handling an entire dataset at once, 2) existence of heterogeneously distributed corruption, and 3) difficulty in corruption estimation when data cannot be entirely loaded. This paper proposes online and distributed robust regression approaches, both of which can concurrently address all the above challenges. Specifically, the distributed algorithm optimizes the regression coefficients of each data block via heuristic hard thresholding and combines all the estimates in a distributed robust consolidation. Furthermore, an online version of the distributed algorithm is proposed to incrementally update the existing estimates with new incoming data. We also prove that our algorithms benefit from strong robustness guarantees in terms of regression coefficient recovery with a constant upper bound on the error of state-of-the-art batch methods. Extensive experiments on synthetic and real datasets demonstrate that our approaches are superior to those of existing methods in effectiveness, with competitive efficiency.

cs.DS↗

arXiv2017

Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank

Recently low displacement rank (LDR) matrices, or so-called structured matrices, have been proposed to compress large-scale neural networks. Empirical results have shown that neural networks with weight matrices of LDR matrices, referred as LDR neural networks, can achieve significant reduction in space and computational complexity while retaining high accuracy. We formally study LDR matrices in deep learning. First, we prove the universal approximation property of LDR neural networks with a mild condition on the displacement operators. We then show that the error bounds of LDR neural networks are as efficient as general neural networks with both single-layer and multiple-layer structure. Finally, we propose back-propagation based training algorithm for general LDR neural networks.

cs.LG↗

arXiv2017

Distributed Least-Squares Iterative Methods in Networks: A Survey

Many science and engineering applications involve solving a linear least-squares system formed from some field measurements. In the distributed cyber-physical systems (CPS), often each sensor node used for measurement only knows partial independent rows of the least-squares system. To compute the least-squares solution they need to gather all these measurement at a centralized location and then compute the solution. These data collection and computation are inefficient because of bandwidth and time constraints and sometimes are infeasible because of data privacy concerns. Thus distributed computations are strongly preferred or demanded in many of the real world applications e.g.: smart-grid, target tracking etc. To compute least squares for the large sparse system of linear equation iterative methods are natural candidates and there are a lot of studies regarding this, however, most of them are related to the efficiency of centralized/parallel computations while and only a few are explicitly about distributed computation or have the potential to apply in distributed networks. This paper surveys the representative iterative methods from several research communities. Some of them were not originally designed for this need, so we slightly modified them to suit our requirement and maintain the consistency. In this survey, we sketch the skeleton of the algorithm first and then analyze its time-to-completion and communication cost. To our best knowledge, this is the first survey of distributed least-squares in distributed networks.

cs.DC↗

arXiv2017

Unsupervised Learning Layers for Video Analysis

This paper presents two unsupervised learning layers (UL layers) for label-free video analysis: one for fully connected layers, and the other for convolutional ones. The proposed UL layers can play two roles: they can be the cost function layer for providing global training signal; meanwhile they can be added to any regular neural network layers for providing local training signals and combined with the training signals backpropagated from upper layers for extracting both slow and fast changing features at layers of different depths. Therefore, the UL layers can be used in either pure unsupervised or semi-supervised settings. Both a closed-form solution and an online learning algorithm for two UL layers are provided. Experiments with unlabeled synthetic and real-world videos demonstrated that the neural networks equipped with UL layers and trained with the proposed online learning algorithm can extract shape and motion information from video sequences of moving objects. The experiments demonstrated the potential applications of UL layers and online learning algorithm to head orientation estimation and moving object localization.

cs.LG↗

arXiv2017

Convergence of ground state solutions for nonlinear Schrödinger equations on graphs

We consider the nonlinear Schrödinger equation $-Δu+(λa(x)+1)u=|u|^{p-1}u$ on a locally finite graph $G=(V,E)$. We prove via the Nehari method that if $a(x)$ satisfies certain assumptions, for any $λ>1$, the equation admits a ground state solution $u_λ$. Moreover, as $λ\rightarrow \infty$, the solution $u_λ$ converges to a solution of the Dirichlet problem $-Δu+u=|u|^{p-1}u$ which is defined on the potential well $Ω$. We also provide a numerical experiment which solves the equation on a finite graph to illustrate our results.

math.AP↗

arXiv2017

Numerically Safe Gaussian Elimination with No Pivoting

Gaussian elimination with no pivoting and block Gaussian elimination are attractive alternatives to the customary but communication intensive Gaussian elimination with partial pivoting (hereafter we use the acronyms GENP, BGE, and GEPP} provided that the computations proceed safely and numerically safely}, that is, run into neither division by 0 nor numerical problems. Empirically, safety and numerical safety of GENP have been consistently observed in a number of papers where an input matrix was pre-processed with various structured multipliers chosen ad hoc. Our present paper provides missing formal support for this empirical observation and explains why it was elusive so far. Namely we prove that GENP is numerically unsafe for a specific class of input matrices in spite of its pre-processing with some well-known and well-tested structured multipliers, but we also prove that GENP and BGE are safe and numerically safe for the average input matrix pre-processed with any nonsingular and well-conditioned multiplier. This should embolden search for sparse and structured multipliers, and we list and test some new classes of them. We also seek randomized pre-processing that universally (that is, for all input matrices) supports (i) safe GENP and BGE with probability 1 and/or (ii) numerically safe GENP and BGE with a probability close to 1.We achieve goal (i) with a Gaussian structured multiplier and goal (ii) with a Gaussian unstructured multiplier and alternatively with Gaussian structured augmentation. We consistently confirm all these formal results with our tests of GENP for benchmark inputs. We have extended our approach to other fundamental matrix computations and keep working on further extensions.

cs.SC↗

arXiv2017

Real Polynomial Root-finding by Means of Matrix and Polynomial Iterations

Univariate polynomial root-finding is a classical subject, still important for modern computing. Frequently one seeks just the real roots of a polynomial with real coefficients. They can be approximated at a low computational cost if the polynomial has no nonreal roots, but for high degree polynomials, nonreal roots are typically much more numerous than the real ones. The challenge is known for a long time, and the subject has been intensively studied. Nevertheless, we produce some novel ideas and techniques and obtain dramatic acceleration of the known algorithms. In order to achieve our progress we exploit the correlation between the computations with matrices and polynomials, randomized matrix computations, and complex plane geometry, extend the techniques of the matrix sign iterations, and use the structure of the companion matrix of the input polynomial. The results of our extensive tests with benchmark polynomials and random matrices are quite encouraging. In particular in our tests the number of iterations required for convergence of our algorithms grew very slowly (if at all) as we increased the degree of the univariate input polynomials and the dimension of the input matrices from 64 to 1024.

cs.SC↗

arXiv2017

Analysis on Cohort Effects in view of Differential Geometry and its Applications

This paper analyzes birth cohort effects and develops an approach which is based on differential geometry to identify and measure cohort effects in mortality data sets. The measurement is quantitative and provides a potential method to compare cohort effects among different countries or groups. Data sets of four countries (e.g. U.k., U.S., Canada and Japan) are taken as examples to explain our approach and applications of the measurement to a modified Lee-Carter model are analyzed. In fact, this paper is an upgrade version of our paper arXiv:1504.00327. There is a new section which gives applications of our approach based on the Lee-Carter and APC models.

stat.AP↗

arXiv2017

Probing the spin-polarized electronic band structure in monolayer transition metal dichalcogenides by optical spectroscopy

We study the electronic band structure in the K/K' valleys of the Brillouin zone of monolayer WSe2 and MoSe2 by optical reflection and photoluminescence spectroscopy on dual-gated field-effect devices. Our experiment reveals the distinct spin polarization in the conduction bands of these compounds by a systematic study of the doping dependence of the A and B excitonic resonances. Electrons in the highest-energy valence band and the lowest-energy conduction band have antiparallel spins in monolayer WSe2, and parallel spins in monolayer MoSe2. The spin splitting is determined to be hundreds of meV for the valence bands and tens of meV for the conduction bands, which are in good agreement with first principles calculations. These values also suggest that both n- and p-type WSe2 and MoSe2 can be relevant for spin- and valley-based applications

cond-mat.mes-hall↗