arXiv subjects

Xingyu Liu

Publications and source records attributed to Xingyu Liu.

97 records · Page 6Linked to original sources

arXiv2019

FlowNet3D: Learning Scene Flow in 3D Point Clouds

Many applications in robotics and human-computer interaction can benefit from understanding 3D motion of points in a dynamic environment, widely noted as scene flow. While most previous methods focus on stereo and RGB-D images as input, few try to estimate scene flow directly from point clouds. In this work, we propose a novel deep neural network named $FlowNet3D$ that learns scene flow from point clouds in an end-to-end fashion. Our network simultaneously learns deep hierarchical features of point clouds and flow embeddings that represent point motions, supported by two newly proposed learning layers for point sets. We evaluate the network on both challenging synthetic data from FlyingThings3D and real Lidar scans from KITTI. Trained on synthetic data only, our network successfully generalizes to real scans, outperforming various baselines and showing competitive results to the prior art. We also demonstrate two applications of our scene flow output (scan registration and motion segmentation) to show its potential wide use cases.

cs.CV↗

arXiv2019

Learning Video Representations from Correspondence Proposals

Correspondences between frames encode rich information about dynamic content in videos. However, it is challenging to effectively capture and learn those due to their irregular structure and complex dynamics. In this paper, we propose a novel neural network that learns video representations by aggregating information from potential correspondences. This network, named $CPNet$, can learn evolving 2D fields with temporal consistency. In particular, it can effectively learn representations for videos by mixing appearance and long-range motion with an RGB-only input. We provide extensive ablation experiments to validate our model. CPNet shows stronger performance than existing methods on Kinetics and achieves the state-of-the-art performance on Something-Something and Jester. We provide analysis towards the behavior of our model and show its robustness to errors in proposals.

cs.CV↗

arXiv2019

Tumor Microenvironment-based Gene Signatures Divides Novel Immune and Stromal Subgroup Classification of Lung Adenocarcinoma

Tumor microenvironment has complex effects on tumorigenesis and metastasis. However, there is still a lack of comprehensive understanding of the relationship among molecular and cellular characteristics in tumor microenvironment, clinical prognosis and immunotherpy response. In this study, the immune and stromal (non-immune) signatures of tumor microenvironment were integrated to identify novel subgroups of lung adenocarcinoma by eigendecomposition and extraction algorithms of bioinformatics and machine learning, such as non-negative matrix factorization and multitask learning. Tumors were classified into 4 groups according to the activation of immunity and stroma by novel signatures. The 4 groups had different mutation landscape, molecular, cellular characteristics and prognosis, which have been validation in 6 independent data sets containing 1551 patients. High-immune and low-stromal activation group links to high immunocyte infiltration, high immunocompetence, low fibroblasts, endothelial cells, collagen, laminin, tumor mutation burden, and better overall survival. We developed a novel model based on tumor microenvironment by integrating immune and stromal activation, namely PMBT (prognostic model based on tumor microenvironment). The PMBT showed the value to predict overall survival and immunotherapy responses.

q-bio.GN↗

arXiv2018

On the Possibility of Singlet Fission in Crystalline Quaterrylene

Singlet fission (SF), the spontaneous down-conversion of a singlet exciton into two triplet excitons residing on neighboring molecules, is a promising route to improving organic photovoltaic (OPV) device efficiencies by harvesting two charge carriers from one photon. However, only a few materials have been discovered that exhibit intermolecular SF in the solid state, most of which are acene derivatives. Recently, there has been a growing interest in rylenes as potential SF materials. We use many-body perturbation theory in the GW approximation and the Bethe-Salpeter equation (BSE) to investigate the possibility of intermolecular SF in crystalline perylene and quaterrylene. A new method is presented for determining the percent charge transfer character (%CT) of an exciton wave-function from double-Bader analysis. This enables relating exciton probability distributions to crystal packing. Based on comparison to known and predicted SF materials with respect to the energy conservation criterion (ES-2ET) and %CT, crystalline quaterrylene is a promising candidate for intermolecular SF. Furthermore, quaterrylene is attractive for OPV applications thanks to its high stability and narrow optical gap. Perylene is not expected to exhibit SF, however it is a promising candidate for harvesting sub-gap photons by triplet-triplet annihilation.

cond-mat.mtrl-sci↗

arXiv2018

Efficient Sparse-Winograd Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices. Their energy is dominated by the number of multiplies needed to perform the convolutions. Winograd's minimal filtering algorithm (Lavin, 2015) and network pruning (Han et al., 2015) can reduce the operation count, but these two methods cannot be directly combined $-$ applying the Winograd transform fills in the sparsity in both the weights and the activations. We propose two modifications to Winograd-based CNNs to enable these methods to exploit sparsity. First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations. Second, we prune the weights in the Winograd domain to exploit static weight sparsity. For models on CIFAR-10, CIFAR-100 and ImageNet datasets, our method reduces the number of multiplications by $10.4\times$, $6.8\times$ and $10.8\times$ respectively with loss of accuracy less than $0.1\%$, outperforming previous baselines by $2.0\times$-$3.0\times$. We also show that moving ReLU to the Winograd domain allows more aggressive pruning.

cs.CV↗

arXiv2017

Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as well as the prediction accuracy. Coarse-grained pruning creates regular sparsity patterns, making it more amenable for hardware acceleration but more challenging to maintain the same accuracy. In this paper we quantitatively measure the trade-off between sparsity regularity and prediction accuracy, providing insights in how to maintain accuracy while having more a more structured sparsity pattern. Our experimental results show that coarse-grained pruning can achieve a sparsity ratio similar to unstructured pruning without loss of accuracy. Moreover, due to the index saving effect, coarse-grained pruning is able to obtain a better compression ratio than fine-grained sparsity at the same accuracy threshold. Based on the recent sparse convolutional neural network accelerator (SCNN), our experiments further demonstrate that coarse-grained sparsity saves about 2x the memory references compared to fine-grained sparsity. Since memory reference is more than two orders of magnitude more expensive than arithmetic operations, the regularity of sparse structure leads to more efficient hardware design.

cs.LG↗

arXiv2016

EIE: Efficient Inference Engine on Compressed Deep Neural Network

State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources and power budgets. While custom hardware helps the computation, fetching weights from DRAM is two orders of magnitude more expensive than ALU operations, and dominates the required power. Previously proposed 'Deep Compression' makes it possible to fit large DNNs (AlexNet and VGGNet) fully in on-chip SRAM. This compression is achieved by pruning the redundant connections and having multiple connections share the same weight. We propose an energy efficient inference engine (EIE) that performs inference on this compressed network model and accelerates the resulting sparse matrix-vector multiplication with weight sharing. Going from DRAM to SRAM gives EIE 120x energy saving; Exploiting sparsity saves 10x; Weight sharing gives 8x; Skipping zero activations from ReLU saves another 3x. Evaluated on nine DNN benchmarks, EIE is 189x and 13x faster when compared to CPU and GPU implementations of the same DNN without compression. EIE has a processing power of 102GOPS/s working directly on a compressed network, corresponding to 3TOPS/s on an uncompressed network, and processes FC layers of AlexNet at 1.88x10^4 frames/sec with a power dissipation of only 600mW. It is 24,000x and 3,400x more energy efficient than a CPU and GPU respectively. Compared with DaDianNao, EIE has 2.9x, 19x and 3x better throughput, energy efficiency and area efficiency.

cs.CV↗