arXiv subjects

Li Zhang

Publications and source records attributed to Li Zhang.

At least 793 records · Page 44Linked to original sources

arXiv2020

What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective

Recent works have demonstrated that global covariance pooling (GCP) has the ability to improve performance of deep convolutional neural networks (CNNs) on visual classification task. Despite considerable advance, the reasons on effectiveness of GCP on deep CNNs have not been well studied. In this paper, we make an attempt to understand what deep CNNs benefit from GCP in a viewpoint of optimization. Specifically, we explore the effect of GCP on deep CNNs in terms of the Lipschitzness of optimization loss and the predictiveness of gradients, and show that GCP can make the optimization landscape more smooth and the gradients more predictive. Furthermore, we discuss the connection between GCP and second-order optimization for deep CNNs. More importantly, above findings can account for several merits of covariance pooling for training deep CNNs that have not been recognized previously or fully explored, including significant acceleration of network convergence (i.e., the networks trained with GCP can support rapid decay of learning rates, achieving favorable performance while significantly reducing number of training epochs), stronger robustness to distorted examples generated by image corruptions and perturbations, and good generalization ability to different vision tasks, e.g., object detection and instance segmentation. We conduct extensive experiments using various deep CNN models on diversified tasks, and the results provide strong support to our findings.

cs.CV↗

arXiv2020

Feedback Graph Convolutional Network for Skeleton-based Action Recognition

Skeleton-based action recognition has attracted considerable attention in computer vision since skeleton data is more robust to the dynamic circumstance and complicated background than other modalities. Recently, many researchers have used the Graph Convolutional Network (GCN) to model spatial-temporal features of skeleton sequences by an end-to-end optimization. However, conventional GCNs are feedforward networks which are impossible for low-level layers to access semantic information in the high-level layers. In this paper, we propose a novel network, named Feedback Graph Convolutional Network (FGCN). This is the first work that introduces the feedback mechanism into GCNs and action recognition. Compared with conventional GCNs, FGCN has the following advantages: (1) a multi-stage temporal sampling strategy is designed to extract spatial-temporal features for action recognition in a coarse-to-fine progressive process; (2) A dense connections based Feedback Graph Convolutional Block (FGCB) is proposed to introduce feedback connections into the GCNs. It transmits the high-level semantic features to the low-level layers and flows temporal information stage by stage to progressively model global spatial-temporal features for action recognition; (3) The FGCN model provides early predictions. In the early stages, the model receives partial information about actions. Naturally, its predictions are relatively coarse. The coarse predictions are treated as the prior to guide the feature learning of later stages for a accurate prediction. Extensive experiments on the datasets, NTU-RGB+D, NTU-RGB+D120 and Northwestern-UCLA, demonstrate that the proposed FGCN is effective for action recognition. It achieves the state-of-the-art performance on the three datasets.

cs.CV↗

arXiv2020

Spin-Valley Locking Effect in Defect States of Monolayer MoS$_2$

Valley pseudospin in two-dimensional (2D) transition-metal dichalcogenides (TMDs) allows optical control of spin-valley polarization and intervalley quantum coherence. Defect states in TMDs give rise to new exciton features and theoretically exhibit spin-valley polarization; however, experimental achievement of this phenomenon remains challenges. Here, we report unambiguous valley pseudospin of defect-bound localized excitons in CVD-grown monolayer MoS2; enhanced valley Zeeman splitting with an effective g-factor of -6.2 is observed. Our results reveal that all five d-orbitals and the increased effective electron mass contribute to the band shift of defect states, demonstrating a new physics of the magnetic responses of defect-bound localized excitons, strikingly different from that of A excitons. Our work paves the way for the manipulation of the spin-valley degrees of freedom through defects toward valleytronic devices.

cond-mat.mtrl-sci↗

arXiv2020

Semantic Discord: Finding Unusual Local Patterns for Time Series

Finding anomalous subsequence in a long time series is a very important but difficult problem. Existing state-of-the-art methods have been focusing on searching for the subsequence that is the most dissimilar to the rest of the subsequences; however, they do not take into account the background patterns that contain the anomalous candidates. As a result, such approaches are likely to miss local anomalies. We introduce a new definition named \textit{semantic discord}, which incorporates the context information from larger subsequences containing the anomaly candidates. We propose an efficient algorithm with a derived lower bound that is up to 3 orders of magnitude faster than the brute force algorithm in real world data. We demonstrate that our method significantly outperforms the state-of-the-art methods in locating anomalies by extensive experiments. We further explain the interpretability of semantic discord.

cs.LG↗

arXiv2020

Enhanced Valley Zeeman Splitting in Fe-Doped Monolayer MoS2

The Zeeman effect offers unique opportunities for magnetic manipulation of the spin degree of freedom (DOF). Recently, valley Zeeman splitting, referring to the lifting of valley degeneracy, has been demonstrated in two-dimensional transition metal dichalcogenides (TMDs) at liquid helium temperature. However, to realize the practical applications of valley pseudospins, the valley DOF must be controllable by a magnetic field at room temperature, which remains a significant challenge. Magnetic doping in TMDs can enhance the Zeeman splitting, however, to achieve this experimentally is not easy. Here, we report unambiguous magnetic manipulation of valley Zeeman splitting at 300 K (g = -6.4) and 10 K (g = -11) in a CVD-grown Fe-doped MoS2 monolayer; the effective g factor can be tuned to -20.7 by increasing the Fe dopant concentration, which represents an approximately fivefold enhancement as compared to undoped MoS2. Our measurements and calculations reveal that the enhanced splitting and geff factors are due to the Heisenberg exchange interaction of the localized magnetic moments (Fe 3d electrons) with MoS2 through the d-orbital hybridization.

cond-mat.mtrl-sci↗

arXiv2020

Superbloom: Bloom filter meets Transformer

We extend the idea of word pieces in natural language models to machine learning tasks on opaque ids. This is achieved by applying hash functions to map each id to multiple hash tokens in a much smaller space, similarly to a Bloom filter. We show that by applying a multi-layer Transformer to these Bloom filter digests, we are able to obtain models with high accuracy. They outperform models of a similar size without hashing and, to a large degree, models of a much larger size trained using sampled softmax with the same computational budget. Our key observation is that it is important to use a multi-layer Transformer for Bloom filter digests to remove ambiguity in the hashed input. We believe this provides an alternative method to solving problems with large vocabulary size.

cs.LG↗

arXiv2020

Selective Convolutional Network: An Efficient Object Detector with Ignoring Background

It is well known that attention mechanisms can effectively improve the performance of many CNNs including object detectors. Instead of refining feature maps prevalently, we reduce the prohibitive computational complexity by a novel attempt at attention. Therefore, we introduce an efficient object detector called Selective Convolutional Network (SCN), which selectively calculates only on the locations that contain meaningful and conducive information. The basic idea is to exclude the insignificant background areas, which effectively reduces the computational cost especially during the feature extraction. To solve it, we design an elaborate structure with negligible overheads to guide the network where to look next. It's end-to-end trainable and easy-embedding. Without additional segmentation datasets, we explores two different train strategies including direct supervision and indirect supervision. Extensive experiments assess the performance on PASCAL VOC2007 and MS COCO detection datasets. Results show that SSD and Pelee integrated with our method averagely reduce the calculations in a range of 1/5 and 1/3 with slight loss of accuracy, demonstrating the feasibility of SCN.

cs.CV↗

arXiv2020

Automatic marker-free registration of tree point-cloud data based on rotating projection

Point-cloud data acquired using a terrestrial laser scanner (TLS) play an important role in digital forestry research. Multiple scans are generally used to overcome occlusion effects and obtain complete tree structural information. However, it is time-consuming and difficult to place artificial reflectors in a forest with complex terrain for marker-based registration, a process that reduces registration automation and efficiency. In this study, we propose an automatic coarse-to-fine method for the registration of point-cloud data from multiple scans of a single tree. In coarse registration, point clouds produced by each scan are projected onto a spherical surface to generate a series of two-dimensional (2D) images, which are used to estimate the initial positions of multiple scans. Corresponding feature-point pairs are then extracted from these series of 2D images. In fine registration, point-cloud data slicing and fitting methods are used to extract corresponding central stem and branch centers for use as tie points to calculate fine transformation parameters. To evaluate the accuracy of registration results, we propose a model of error evaluation via calculating the distances between center points from corresponding branches in adjacent scans. For accurate evaluation, we conducted experiments on two simulated trees and a real-world tree. Average registration errors of the proposed method were 0.26m around on simulated tree point clouds, and 0.05m around on real-world tree point cloud.

cs.CV↗

arXiv2020

Searching for Quasi-Periodic Modulations in $γ$-ray Active Galactic Nuclei

We perform a systematic search of quasi-periodic variabilities in $γ$-ray active galactic nuclei (AGNs) in the third \emph{Fermi} Large Area Telescope source catalog (3FGL). We employe two techniques, Lomb-Scargle Periodogram (LSP) and Weighted Wavelet Z-transform (WWZ), to obtain power spectra of $γ$-ray light curves covering from 2008 August to 2016 December. The results show that besides several objects that have been reported in previous works, an additional source, the FSRQ PKS 0601-70 has a possible quasi-periodic variability of 450 days in its $γ$-ray light curves with the significance of $>3σ$. The physical implications of our findings are discussed.

astro-ph.HE↗

arXiv2020

Estimating the Unruh effect via entangled many-body probes

We study the estimation of parameters in a quantum metrology scheme based on entangled many-body Unruh-DeWitt detectors. It is found that the precision for the estimation of Unruh effect can be enhanced via initial state preparations and parameter selections. It is shown that the precision in the estimation of the Unruh temperature in terms of a many-body-probe metrology is always better than the precision in two probe strategies. The proper acceleration for Bob's detector and the interaction between the accelerated detector and the external field have significant influences on the precision for the Unruh effect's estimation. In addition, the probe state prepared with more excited atoms in the initial state is found to perform better than less excited initial states. However, different from the estimation of the Unruh temperature, the estimation of the effective coupling parameter for the accelerated detector requires more total atoms but less excited atoms in the estimations.

gr-qc↗

arXiv2020

Depth creates no more spurious local minima

We show that for any convex differentiable loss, a deep linear network has no spurious local minima as long as it is true for the two layer case. This reduction greatly simplifies the study on the existence of spurious local minima in deep linear networks. When applied to the quadratic loss, our result immediately implies the powerful result in [Kawaguchi 2016]. Further, with the work in [Zhou and Liang 2018], we can remove all the assumptions in [Kawaguchi 2016]. This property holds for more general "multi-tower" linear networks too. Our proof builds on [Laurent and von Brecht 2018] and develops a new perturbation argument to show that any spurious local minimum must have full rank, a structural property which can be useful more generally.

cs.LG↗

arXiv2019

Truncated analytic moment analysis and its hybrid-field contrast in grating-based x-ray phase contrast imaging

For grating-based x-ray phase contrast imaging (GPCI), a multi-order moment analysis (MMA) has been recently developed to obtain multiple contrasts from the ultra-small-angle x-ray scattering distribution, as a novel information retrieval approach that is totally different from the conventional Fourier components analysis (FCA). In this paper, we present an analytic form of MMA in theory that can retrieve multiple contrasts directly from raw phase-stepping images, with no scattering distribution involved. For practical implementation, a truncated analytic analysis (called as TA-MMA) is adopted and it is hundreds of times faster in computation than the original deconvolution-based MMA (called as DB-MMA). More importantly, TA-MMA is proved to establish a quantitative connection between FCA and MMA, i.e., the first-order moment computed by TA-MMA is essentially the product of the phase contrast and the dark-field contrast retrieved by FCA, providing a new physical parameter for GPCI. The new physical parameter, in fact, can be treated as a ``hybrid-field'' contrast as it fuses the original phase contrast and dark-field contrast in a straightforward manner, which may be the first physical fusion contrast in GPCI to our knowledge and may have a potential to be directly used in practical applications.

physics.med-ph↗

arXiv2019

Deep Self-representative Concept Factorization Network for Representation Learning

In this paper, we investigate the unsupervised deep representation learning issue and technically propose a novel framework called Deep Self-representative Concept Factorization Network (DSCF-Net), for clustering deep features. To improve the representation and clustering abilities, DSCF-Net explicitly considers discovering hidden deep semantic features, enhancing the robustness proper-ties of the deep factorization to noise and preserving the local man-ifold structures of deep features. Specifically, DSCF-Net seamlessly integrates the robust deep concept factorization, deep self-expressive representation and adaptive locality preserving feature learning into a unified framework. To discover hidden deep repre-sentations, DSCF-Net designs a hierarchical factorization architec-ture using multiple layers of linear transformations, where the hierarchical representation is performed by formulating the prob-lem as optimizing the basis concepts in each layer to improve the representation indirectly. DSCF-Net also improves the robustness by subspace recovery for sparse error correction firstly and then performs the deep factorization in the recovered visual subspace. To obtain locality-preserving representations, we also present an adaptive deep self-representative weighting strategy by using the coefficient matrix as the adaptive reconstruction weights to keep the locality of representations. Extensive comparison results with several other related models show that DSCF-Net delivers state-of-the-art performance on several public databases.

cs.LG↗

arXiv2019

Valley kink states and topological channel intersections in substrate-integrated photonic circuitry

Valley degrees of freedom, providing a novel way to increase capacity and efficiency of information processing, have become an important instrument for photonics. Experimental studies on photonic topological valley kink states at interfaces between regions with opposite valley-Chern numbers have attracted much attention, however, were restricted to zigzag-type interfaces, largely limiting their applications such as geometry-dependent topological channel intersections. Here, we experimentally demonstrate and manipulate valley kink states at generic interfaces in subwavelength substrate-integrated photonic circuitry. We verify the robustness of the kink states by measuring transmissions of the kink states through twisted interfaces and interfaces with disorders. Based on the valley kink states at generic interfaces, we realize several topological channel intersections where photonic transport paths are related to geometries of the intersections. In comparison to those in previous works, the present valley photonic crystals have subwavelength thicknesses and excellent self-consistent electrical shielding, which are perfectly compatible with conventional substrate-integrated photonic circuitry. Our work opens a door to manipulate photonic valley pseudo-spins in substrate-integrated circuitry with robustness, easy access, and lightweight.

physics.app-ph↗

arXiv2019

Discriminative Local Sparse Representation by Robust Adaptive Dictionary Pair Learning

In this paper, we propose a structured Robust Adaptive Dic-tionary Pair Learning (RA-DPL) framework for the discrim-inative sparse representation learning. To achieve powerful representation ability of the available samples, the setting of RA-DPL seamlessly integrates the robust projective dictionary pair learning, locality-adaptive sparse representations and discriminative coding coefficients learning into a unified learning framework. Specifically, RA-DPL improves existing projective dictionary pair learning in four perspectives. First, it applies a sparse l2,1-norm based metric to encode the recon-struction error to deliver the robust projective dictionary pairs, and the l2,1-norm has the potential to minimize the error. Sec-ond, it imposes the robust l2,1-norm clearly on the analysis dictionary to ensure the sparse property of the coding coeffi-cients rather than using the costly l0/l1-norm. As such, the robustness of the data representation and the efficiency of the learning process are jointly considered to guarantee the effi-cacy of our RA-DPL. Third, RA-DPL conceives a structured reconstruction weight learning paradigm to preserve the local structures of the coding coefficients within each class clearly in an adaptive manner, which encourages to produce the locality preserving representations. Fourth, it also considers improving the discriminating ability of coding coefficients and dictionary by incorporating a discriminating function, which can ensure high intra-class compactness and inter-class separation in the code space. Extensive experiments show that our RA-DPL can obtain superior performance over other state-of-the-arts.

cs.CV↗

arXiv2019

Synchronize accelerated clock in a multipartite relativistic quantum system

We perform a protocol for multipartite quantum clock synchronization under the influence of Unruh thermal noise. The clocks consisting of Unruh-DeWitt detectors when one of detectors accelerated is obtained. To estimate the time difference between the clocks, we calculate the time probability and analyze how the probability is influenced by the Unruh thermal noise and other factors. It is shown that both relativistic motion and interaction between the atom and the external scalar field affect the choice of optimal number of excited atoms in the initial state, which leads to a higher clock adjustment accuracy. Time probabilities for different types of initial states approach to the same value in the limit of infinite acceleration, while tend to different minimums with increasing number of atoms. In addition, the accuracy of clock synchronization using a pair of entangled clocks in two-party system is always higher than in an multipartite system, while the optimal $Z$-type multipartite initial state always perform better than the $W$-type state.

quant-ph↗

arXiv2019

PGU-net+: Progressive Growing of U-net+ for Automated Cervical Nuclei Segmentation

Automated cervical nucleus segmentation based on deep learning can effectively improve the quantitative analysis of cervical cancer. However, accurate nuclei segmentation is still challenging. The classic U-net has not achieved satisfactory results on this task, because it mixes the information of different scales that affect each other, which limits the segmentation accuracy of the model. To solve this problem, we propose a progressive growing U-net (PGU-net+) model, which uses two paradigms to extract image features at different scales in a more independent way. First, we add residual modules between different scales of U-net, which enforces the model to learn the approximate shape of the annotation in the coarser scale, and to learn the residual between the annotation and the approximate shape in the finer scale. Second, we start to train the model with the coarsest part and then progressively add finer part to the training until the full model is included. When we train a finer part, we will reduce the learning rate of the previous coarser part, which further ensures that the model independently extracts information from different scales. We conduct several comparative experiments on the Herlev dataset. The experimental results show that the PGU-net+ has superior accuracy than the previous state-of-the-art methods on cervical nuclei segmentation.

eess.IV↗

arXiv2019

Layer Dependence of Stacking Order in Nonencapsulated Few-layer CrI3

Long-range magnetic orders in atomically thin ferromagnetic CrI3 give rise to new fascinating physics and application perspectives. The physical properties of two-dimensional (2D) ferromagnetism CrI3 are significantly influenced by interlayer spacing and stacking order, which are sensitive to the hydrostatic pressure and external environments. However, there remains debate on the stacking order at low temperature. Here, we study the interlayer coupling and stacking order of non-encapsulated 2-5 layer and bulk CrI3 at 10 K by Raman spectroscopy; demonstrate a rhombohedral stacking in both antiferromagnetic and ferromagnetic CrI3. The opposite helicity dependence of Ag and Eg modes arising from phonon symmetry further validate the rhombohedral stacking. An anomalous temperature-dependent behavior is observed due to spin-phonon coupling below 60 K. Our work provides insights into the interlayer coupling and stacking orders of 2D ferromagnetic materials.

cond-mat.mtrl-sci↗